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Robert M. Fano 
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setts Institute of Technology, where he worked on 
microwave components and filters. Except for this 
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Electrical Engineering Department at M.I.T. since 
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of network theory, electromagnetic theory, and 
information theory. He is co-author of Volume 9 of 
the Radiation Laboratory Series, and he has just 
completed the manuscript of a book on electro- 
magnetic theory in collaboration with Dr. L. J. Chu. 

He is a Fellow of the Institute of Radio Engineers, 
a Fellow of the American Academy of Arts and 
Science, and a member of Sigma Xi, Eta Kappa Nu, 
and other professional societies. He has been a 
member of the Administrative Committee of the 
PGIT ever since its formation and is presently 
Chairman of the Editorial Board of the Trans- 
ACTIONS of the PGIT. 
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The Challenge of Digital Communication 


ROBERT M. FANO 


The necessity for transmitting digital information with 
ry low probability of error is becoming a major com- 
unication problem. As is often the case in technical 
velopments, this problem is arising just at the time 
hen the basic theoretical tools for solving it are beginning 
_ take shape. A new challenge for communication 
ineers is being born. 

The development of high-speed digital computers is 
aking possible the analysis and processing in real time 

increasingly large amounts of data. With their help, 

e detailed coordination of complex activities at widely 
parated locations is becoming technically feasible. 
ir-traffic control is an example of this. Thus, we can 
pect that in a relatively few years a major part of 
munication traffic will consist of information generated 
y, or to be processed by, digital computers. By contrast, 

the past, practically all communication traffic was 
nerated by and directed to humans. A review of the 
lequacy of our communication techniques for such a 
dically different task is in order. 

Communication between humans is greatly aided by 
ie use of natural encoding schemes such as spoken 
nguage, developed by evolution to combat the various 
sturbances in the normal human environment. Above all, 
ynmunication between humans hinges on the decoding 
yeration performed by the brain of the receiver, which 
in recognize the transmitted message with a high 
sgree of reliability even in the presence of substantial, 
npredictable disturbances. Thus, when a human is the 
ceiver of messages transmitted through an electric 
mmmunication system, it is sufficient to maintain the 
vel of the electric disturbances within the rather broad 
mits for which the human coding and decoding system 
un operate satisfactorily. 

Since present-day electric communication systems are 
rimarily designed for direct communication between 
umans they cannot be expected, in general, to perform 
itisfactorily as links between digital computers without 
srminal equipment that is able to perform encoding and 
ecoding operations similar to those performed by the 
uman brain. The analogy can be pushed further by 
isualizing these operations as being performed directly 
y the same digital computers that generate and use the 
formation. 

Information theory throws considerable light on how 
igital information can be transmitted with low prob- 
bility of error. The probability of error per digit depends 
pon three parameters: the capacity of the communication 
nannel, the rate of transmission of information, and the 
ze of the “packages” of information that are encoded 
yr transmission as single units. Shannon has shown that 
1e probability of error after decoding can be reduced 


as much as desired, for fixed channel capacity and fixed 
rate of transmission, provided only that the former 
is greater than the latter. This result is accomplished 
by increasing the size of the packages of information that 
are encoded as single units. lor the purpose of the present 
discussion, each package of information can be thought 
of as being encoded into a sequence of pulses (binary or 
n-ary). Some of them are information pulses that repre- 
sent directly the digits that are to be transmitted; the 
rest are ‘“‘check pulses” uniquely specified by the informa- 
tion pulses. The ratio of the two types of pulses remains 
constant when the size of the package is increased. 

Feinstein, Elias, and Shannon have shown that by 
proper encoding, that is, by proper selection of the rules 
that specify the check pulses, the probability of error per 
digit can be made to decrease exponentially with the 
number of digits in each package. The coefficient in this 
exponent, by which the number of digits is multiplied, 
increases with the ratio of the channel capacity to the 
rate of transmission. 

Thus, the probability of error can be decreased either 
by increasing the channel capacity in relation to the rate 
of transmission, or by increasing the number of digits 
in each package. 

The main technical difficulty that must be solved before 
large packages of information can be encoded as units 
concerns the decoding apparatus. If we attempt to decode 
by following the most obvious procedure, the complexity 
of the decoding operation grows exponentially with the 
size of the information packages at the same time that the 
probability of error decreases exponentially. Thus, for 
example, the complexity would have to be doubled in 
order to reduce the probability of error by a factor of 
two. This would hardly be an attractive exchange. On 
the other hand, some recent work by Wozencraft indicates 
that with more efficient decoding procedures, the com- 
plexity of the decoding operation can be caused to increase 
at a rate that is slower than the square of the size of the 
information packages. Although many questions remain to 
be answered, it appears that the technical difficulties 
involved in the decoding operation are within the manage- 
able range. 

In the past, low probabilities of error have been achieved 
by using a channel capacity much larger than the trans- 
mission rate, with packages of minimum size. Ratios 
as great as ten are often used in binary pulse transmission. 
Great saving of channel capacity could be achieved in 
such cases by increasing the information packages. 
Furthermore, there are situations in which the channel 
capacity that can be made available is insufficient to 
provide an acceptable probability of error for the required 
rate of transmission. 
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While it 1s now technically simpler to use packages of 
minimum size and reduce the probability of error by 
increasing the channel capacity, consideration of long- 
range economic factors points to the other solution. The 
price of channel capacity is not likely to decrease very 
much; it is more likely to increase. In fact, the installation 
of new communication facilities involves a great deal of 
work, such as digging trenches for cables and erecting 
large antennas, which cannot be decreased much further 
by mass production. On the other hand, the equipment 
necessary for coding and decoding, which consist mostly 
of logical circuitry of the digital type, can be readily 
mass-produced once the design problems have been solved. 
Furthermore, the coding and decoding operations can be 
performed by general purpose digital computers. As 
suggested above, they may require only an increase in the 
size or speed of the computers that generate and receive 
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the information. It seems clear that the cost of coding 
and decoding equipment is likely to decrease substantially 
in the future, while the cost of channel capacity is likely t« 
remain of the same order of magnitude. While it is difficul 
to predict exactly where and when the two cost curve 
will cross, it is clear that they will do so in the not toc 
distant future. 

The key problem in achieving efficient transmission 0 
information with very low probability of error lies in th 
decoding operation, which is in effect a form of informatior 
processing. This is one further piece of evidence that the 
fields of information processing and information trans 
mission are moving steadily closer together, and ar¢ 
beginning to interact strongly with each other. Very 
likely we shall see them merge in the next few years int¢ 
a broader field that will challenge the combined ranks o 
computer and communication engineers. 
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Some Comments on the Detection of Gaussian Signals 
in Gaussian Noise’ 


D. SLEPIANT 


Summary—It is pointed out that a frequently used mathematical 
1odel of the detection problem yields detection with arbitrarily 
mall probability of error in many cases of engineering interest. 
ome comments are made as to why the model is inadequate from 
n engineering standpoint. 


I. Tar ComMENTS 


HE problem of detecting a Gaussian signal in 
{ Gaussian noise has been discussed by a number 
of authors during the past decade. A recent paper 
y Middleton’ on this subject contains references to 
wuch of the earlier work. Here we comment on several 
spects of this problem generally overlooked in the past. 

The problem treated can be stated as follows. An 
bserver has available to him a sample of finite duration, 
(t),O <t < T, of a stationary Gaussian process. It is 
nown to him that x(¢) is either a sample from the Gaussian 
nsemble A with mean zero and power-density spectrum 
a(f) or a sample from the Gaussian ensemble B with 
ean zero and power-density spectrum ¢,(f). The observer 
to decide whether x(t), 0 < t < 7 came from A or B. 
n most engineering applications, A is interpreted as 
gnal plus noise and B as noise alone. 

The observer’s decision as to which ensemble x(t) 
ume from can be in error in two different ways: he can 
ssert that x(t) came from A when indeed it came from 
; or he can assert that x(t) came from B when indeed 
-came from A. We denote the probabilities of these two 
ypes of error by p, and pz, respectively. 

The main result of this paper is that for the spectra 
anerally considered in engineering problems, it is possible 
x the observer to make his decision with vanishingly 
nall probability of error of either kind. More specifically, 
e have the following 


‘heorem 


Let x(t), O < t < T, be a sample from a stationary 
aussian ensemble with mean zero and power-density 
gectrum ga(f) or en(f). If 

ga(f) # os(f), 

val(f) ts either bandlimited or rational in f, 

va(f) is either bandlimited or rational in f, and if, in the 
ise where v4, and ¢g, are both rational 


* Manuscript received by the PGIT, November 15, 1957. 
he work reported here was supported in part by the Office of 
aval Research under Contract Nonr 210 (00). 

+ Bell Telephone Labs., Inc., Murray Hill, N. J. 

1], Middleton. ‘On the detection of stochastic signals in 
iditive normal noise—Part I,” IRE Trans. on INFORMATION 
HEORY, vol. IT-3, pp. 86-121; June, 1957. 
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then there exist tests based on x(t), 0 < t < T, such that 
Pa < €and pz < «for any preassigned « > 0. This result 
holds for arbitrarily small T > 0. It is to be noted that 
when A is signal plus noise and B is noise alone, the 
exceptional case in which g4 and ¢g,» are both rational and 
lim;.. ¢4/¢z = 1 can occur only if the signal spectrum 
falls off more rapidly with large f than the noise spectrum. 
This case, for which the theorem makes no statement, is of 
some engineering interest. Examples are given by Middle- 
ton." 

The apparent contradiction of engineering intuition 
and experience implied by this theorem does not arise 
because of some trick hidden in the mathematical problem 
posed, but rather because the mathematical problem 
posed is not a good model of the detection problem 
encountered by the engineer. The model fails in at least 
two major respects even when the processes involved are 
truly Gaussian and stationary. In the first place, in 
practice the spectra g4 and ¢g, are not known exactly to 
the observer. The power spectra “known” to him are the 
results of measurements (frequently crude) or of educated 
guesses. In the second place, the observer does not have 
available to him a sample of either process in the sense 
implied by the mathematical notation x(t), 0 <t < T. 
He cannot, for example, meaningfully differentiate his 
observed sample 3000 times, nor can he meaningfully 
measure the values of the observed sample at time instants 
separated by only 10 "°° seconds. 

It will be seen below that the tests by which the results 
of the theorem can be obtained require the observer to 
make use of arbitrarily fine detail of the observed sample 
and required him to know precisely the behavior of ¢4 
and gs as f > ©. If it is assumed that yg, and ¢ , are 
known precisely to the observer, but that some restriction 
is placed upon his ability to measure the sample x(t), the 
resulting model of the engineering detection problem 
still retains its unrealistic features. For example, if the 
observer has available to him only the quantities xe(j7/n) + 
e;,] = 0,1, --: , n, where the measurement errors €; are 
assumed to be normal quantities of known mean and 
covariance, then in many cases the probabilities of error, 
pa and pz, can still be made as sinall as desired by making 
n large. To make a model of the detection problem 
satisfactory for engineering use it seems necessary to 
assume both 1) incomplete knowledge by the observer of 
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the spectra ¢, and ¢g,; and 2) some restrictions on the 
observer’s ability to measure fine detail of the actual 
process made available to him. We cannot, however, 
enter further into these matters here. 

It is worth pointing out that in some published papers 
on this detection problem, ‘‘best’’ solutions are obtained 
in the case where the observer has available to him the 
quantities x(j7/n), 7 = 0, 1, +--+ , n. Ina manner which 
must be regarded as purely heuristic, limiting forms are 
written down for n — © and are presented as solutions 
to the problem for the case in which the observer measures 
x(t) continuously in 0 < t < T. In view of the foregoing 
theorem, these “continuous solutions’ must be treated 
with some caution. 


II. Proor or THEOREM 


The proof of the theorem follows immediately from 
results available in the literature. If ¢, and ¢, are band- 
limited, then with probability one the sample functions 
of A and B are analytic.” The sample functions can, 
therefore, be expanded in a Taylor series of radius of 
convergence p > 0 about any point. Since the process is 
stationary, p is independent of the point about which the 
expansion is made. The observer can, therefore, obtain 
the values of x(t) everywhere from its values in0 < ¢t < T 
by making repeated Taylor series expansions. The auto- 
correlation of x(t) can then be computed and the Fourier 
transform of this autocorrelation obtained. If x(t) came 
from A, this transform will be g4 with probability one; 
if x(t) came from B, this transform will be ¢, with prob- 
ability one.” 

In the case of rational spectra, we make use of a theorem 
due to Baxter.* As adapted for our purposes, this theorem 
can be stated as follows. Let x(t) be a sample from a 
stationary Gaussian process with mean zero and covaria- 
ance function r(r) = FE x(t)x(t + 7). Let r(7) be con- 
tinuous and have a uniformly bounded second derivative 


for 7 ~ 0. Let 
| jG+ DE br = {in \ } 
n n 


and a —e2i7 (O--), Unen lime. friiy, — ao 
> e} = 0 for every « > 0. That is, the test function (1) 
converges in probability to the jump in the first derivative 
of the covariance function at the origin. 

To apply the theorem to the problem at hand, suppose 
that for large f, ¢, ~ a/f and g, ~ b/f”, p > 1. Let 
r4(r) and rz(7) be the covariance functions of A and B. 
Then — 2 rf(0O+) = (2)? a and — 2r/(0+) is zero if 
p > 1 and is (27)’b if p = 1. The test function y, of (1), 


Ti 


Rea, 


7=0 


(1) 


2M. Loeve. “Probability Theory,” D. Van Nostrand Co., Inc., 
New York, N. Y., corollary 3, p. 471; 1955. 

3 The reasoning here requires A and B to be ergodic. This con- 
dition is fulfilled for the processes under discussion. See U. Grenan- 
der, ‘Stochastic processes and statistical inference,’’ Arkiv. for 
Matematik, band 1, no. 17, theorem, p. 257; 1950. 

‘Glen Baxter, “A strong limit theorem for Gaussian processes,” 
Proc. Amer. Math. Soc., vol. 7, pp. 522-527; June, 1956. 
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then, will have a value close to (2r)*aT with high prob- 
ability when n is large and x(t) did come from A. When 
x(t) came from B, y, will with high probability be close 
to zero if p > 1 and will be close to (27r)*6T’ with high 
probability if p = 1. The rules for the observer can be 
given in the usual form of a threshold test. If p > 1 
choose a threshold, 7, between zero and (2mr)’aT’. If 
Yn > ¥Y, say that x(t) came from A. If y, < g say that 
a(t) came from B. If p = 1, choose a threshold, 7, that 
lies between (27r)’aT and (2r)*bT. If a > b, say that 
a(t) came from A when y, > 9g and say that x(t) came 
from B when y, < g. If b > a, reverse the decision rule. 
The convergence of y, in probability assures us that by 
taking n sufficiently large, both p4 and pg can be made 
arbitrarily small by the above procedure unless possibly 
p = landa = b. 

To adapt Baxter’s theorem to our problem when both 
ra(r) and rz(r) have a continuous first derivative at 
7 = 0, we note’ that if g, ~ a/f’”, then — 2r°"-” (0+) = 
(— 1)” '(2r)’"a, 7.e., the (2m — 1) st derivative of the 
covariance function has a jump at the origin of magnitude 
(2r)*"a. Derivatives of order less than 2m — 1 are con- 
tinuous at 7 0. Furthermore, the process A is dif- 
ferentiable’ m—1 times and (— 1)” r°”"?(r) is the 
covariance function of the derived process x‘” *(r). 

If, then, oy ~ a/f”™ and os ~ b/f 7 “ap = Ome 
only necessary to differentiate x(t) m—1 times before 
applying the test function (1) to obtain the same detection 
results as before. That is, the observer uses 


n—1 


ee { i H 
go ee 
where z(t) = [d” */dt”*] a(t), and decision rules, and 
thresholds analogous to those used in the case m = 1 
already treated. 

This same test will clearly distinguish between the two 
ensembles when ¢, is rational, ¢, ~ a/f’” say, and ¢, 


is band-limited. 


ILL. Some Oruer ForMS oF THE TEST 


As already noted, the detection problem is frequently 
restricted to allow the observer access to only a finite set 
of points, 747 /n), 7 = 0, 1, --- ,,-or in the limit a de 
numerable set of points. The tests described in the preced- 
ing paragraph required, for the most part, using infinitely 
many values of x(t) even for finite values of n, e.g., de- 
rivatives were taken. It is of some interest, then, to show 
that detection with arbitrarily small values of p, and pz, 
is possible using only a finite number of values of a(t). We 
treat here only the case in which ¢,4 and ¢, are rational 
and the observer has access to #(jT/n), 3 = 0,1, --- , n. 
The results obtained are generalizations of Baxter’s 
theorem. 


5 See J. L. Doob, “Stochastic Processes,’ John Wiley and Sons, 
Inc., New York, N. Y., pp. 542-551; 1953, for a discussion of the 
results stated in this paragraph. 

6 Ibid. A separable version of the process is assumed. 
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integers and s > m. Consider 
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(The expected value of y, is 
Wem 1 
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‘On introducing the Fourier integral representation for 
v4, this can be written 


(2) 


Ky, = 


: et yy Os co q 
Ly, am mr? ite fealf) 


a “ {m\fm yy kth 224i (1-k) f(T/n) 
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k=0 1=0 


oem 2m—-1 
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‘i m ie a E) 4 Gal E ; 


It then follows easily from the asymptotic behavior of 
wo, thatifs > m 
| lim Hy, = 0 


q-o 


svhile ifs = m, 


| 


lim BY on 


qn 


= OG 


vhere 
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The variance of y, can also be computed in a straight- 
forward manner. One finds 


Horu=(a) EE cee 


[edie — o)m +k — u] a oO) Mt 1b ¥] z 


H rf(G = yas 1A = 12, TUG a eo ma 


Introduce the Fourier integral representation of r4, 
interchange summation and integration and perform the 
sums. There results 


vary. = 2B) fat [ affoaealt? 
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or, after an appropriate change of variables 


Slepian: Detection of Gaussian Signals in Gaussian Noise 


Let g, ~ a/f* and n = mq where s,m and q are positive 
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We now show that lim,.,.. Var y,,, = 0. Since all factors 
in the integrand of (3) are nonnegative and since f’"v,(f) 
is bounded from above by our assumption, it follows that 


Var You < oy (4) 
where d does not depend on n or gq and 
i “J2m 
B & sin a sin — ; 4 
olf af al "| [soe 
ss EOE MMs, pele) | sin € — 7) 
| om ol 
Now 
CP aE Der (k+1) 7 
hy == Les; Ss ie ae, f dn, 
Y j=-@ k=-@ Jj 
sin Si sin m ; ; 
m E qé, — | 
a Nk sin (£; — m) 
| m m | 
eae dl sin q( 
=1 fia [a] a2 £= 9 Teneo 
-if pe (x) (5) 
a qd 0) eg sin Xx J Z 
where 
[~ : il ; 2m 
« |sin7- (—E + jr) 
he caller 
Sa ites (Em) 
and 
ga) = | dy BY + DBY - 0): 


The square bracket in (5) is the Fejer kernel studied in 
Fourier theory.’ As ¢ > ©, h, > x g(0+) which is finite. 
From (4) it follows that Var y,,, ~ONasq—7 © . 

The preceding paragraphs show that if 4 ~ a/f”, 
then y,,, of (2) converges in probability to zero if s > m 
and to ac,, if s = m. Consider now the problem of dis- 
tinguishing between the stationary Gaussian ensembles 
A and B where g, ~ a/f’” and gs ~ b/f'"*”, p = 0. 
Form the test function (2) from the observed samples 


7E. T. Whittaker and G. N. Watson, “Modern Analysis,” The 
MacMillan Co., New York, N. Y., pp. 170-171; 1947. 
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“QT yn), 9° =.0; 1) -+ + n,n = mg it pe OM chooses: 
threshold 7 between zero and c,,a. Use the decision rule 
“sample came from A if y,,, > 9; sample came from B if 
Yon < Y.”’ If p = 0 and a > B, choose a threshold 7 
between c,,a and c,,b. Use the decision rule ‘‘sample came 
from A if y «n => 9; sample came from B if y,, < g.” If 
p = O anda < b, reverse the decision rule. By choosing 
q, and hence n, sufficiently large p, and pz can be made 
arbitrarily small. 

In the test just described, it was necessary to use 
arbitrarily many values of the observed sample x(t) to 
make p, and pz arbitrarily small. It is interesting to note 
that if ¢, ~ a/f’” and g, ~ b/f'"*”, p > 0, it is possible 
to obtain these same results using only the m + 1 observed 
values x(j77'/n), 7 = 0, 1, --- , m. The test function to be 
used in this case is 


aaa (2) BS (™) (2). 


A threshold 7 is chosen. If y, > 7, the observer says x(t) 
came from A; otherwise that x(t) came from B. It is easy 
to show that by making n large enough and by choosing 
y properly (7 increases with n) pa and pg can be made 
arbitrarily small. We omit the details here. It is curious 
that the total observation time of the process needed to 
make this test becomes vanishingly small as the accuracy 
of the test improves. 

We note that the results of this section are valid for a 
larger class of stationary Gaussian processes than those 
having rational power spectral densities. They are valid, 
for example, for all bounded g, and g, having the pre- 
scribed asymptotic behavior. 


IV. Lixevirmoop Rario TEsts 


Let, the quantities z; = «(jT/n), 7 = 0, 1, -:- , n be 
available to the observer. The x; are Jointly Gaussian 
with mean zero. If x(t) comes from A, the covariance 
matrix of the x; has elements A;; = ra[(¢ — j) (T'/n)]; 
if the x; came from B, their covariance matrix has elements 
B;; = ra{@ — 7) (T/n)]. Let the corresponding Gaussian 
probability densities be denoted by ga(xo, --: , %,) and 
Jp(Xo, , «,). Many desirable tests’ to distinguish 
between A and B have decision rules of the form: choose 
A if the likelihood ratio g4/gsz exceeds a threshold value; 
choose B otherwise. For example, the Neyman-Pearson 
test which minimizes p, for a given value of pz is of this 
form. Since for Gaussian densities, the likelihood ratio 
depends monotonely on the quadratic form 

Yn = : 2 [Aim Bilea;, (6) 
the characteristics of these threshold tests can be de_ 
termined by studying the random variable y,. 

Let fra(y,) and f,»(y,) denote the probability density 

function of (6) according as x(t) came from A or B re- 
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spectively. Then the probabilities of the two kinds of 
decision errors are 


pa = i} : fisly) dy, = pa = i & fraly) dy (7) 


where 7 is the threshold. To investigate the behavior of 
these errors for large n, it is necessary to study f,z(y) 
and fna(y)- 

Such a study appears to be quite difficult for general 
covariance functions r4(7) and r,(7). The results of 
Section III show, however, that by suitable choice of 9, 
pa and pz, can be made arbitrarily small in the case where 
r, and rz, are derived from rational spectra not possessing 
identical asymptotic behavior. For, like (6), the tests 
considered in Section III use only the quantities 2; and 
they lead to arbitrarily small probabilities p4 and pz. 
From the optimum properties of the Neyman-Pearson 
test, (6) must do as well or better for each n. 

Much more can be said in detail about f,4 and f,, mm 
the case where r4 and 7, are rational, but we put this off 
for a possible later paper. We note here only that the 
exclusion of the case in which g4/¢, — 1 is not unnecessary 
in general, for if r, = ae and fe = Racal ah DA 
and pp, of (7) cannot simultaneously be made arbitrarily 
small when a,6; = ario 


V. CONCLUSIONS 


The purely mathematical problem of testing the 
hypothesis that x(t), 0 <¢< 7, came from the stationary 
Gaussian ensemble A with mean zero and known spectral 
density ¢, against the alternate hypothesis that «(¢) 
came from the stationary Gaussian ensemble B with 
mean zero and known spectral density gg poses many 
difficult and interesting questions. In particular, necessary 
and sufficient conditions that the test be made with 
arbitrarily small probability of error are not known.” 
For the class of rational and bandlimited spectra (with 
some minor exceptions), it zs possible to decide with 
arbitrarily small probability of error which ensemble 
x(t) was drawn from. This mathematical hypothesis- 
testing problem is, in general, a poor model of the de- 
tection problem faced by the engineer. An adequate 
mathematical model of the detection problem must not 
assume perfect a priort knowledge of ¢, and gg and must 
place limitations on the observer’s ability to measure 


ee 
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made arbitrarily small if o? 4 1. 
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A Useful Theorem for Nonlinear Devices 
Having Gaussian Inputs’ 


ROBERT PRICET 


Summary—tIf and only if the inputs to a set of nonlinear, zero- 
memory devices are variates drawn from a Gaussian random 
process, a useful general relationship may be found between 
certain input and output statistics of the set. This relationship 
equates partial derivatives of the (high-order) output correlation 
coefficient taken with respect to the input correlation coefficients, 

_ to the output correlation coefficient of a new set of nonlinear devices 
_ bearing a simple derivative relation to the original set. Application 
is made to the interesting special cases of conventional cross- 
correlation and autocorrelation functions, and Bussgang’s theorem 
is easily proved. As examples, the output autocorrelation functions 
are simply obtained for a hard limiter, linear detector, clipper, 
and smooth limiter. 


N THE COURSE of investigating the asymptotic 
if frequeney behavior of power spectra resulting from 

the passage of noise through zero-memory nonlinear 
devices, an interesting, unique property of Gaussian 
processes has been encountered, which does not appear 
_to have been previously reported. 


STATEMENT OF THE THEOREM 
| 

| Assume 2, 22, -:: , 2% to be random variables from a 
Gaussian process whose nth order joint probability density 


“is given by:’ 


; tp) = (20) "| M, |” 


n n M 
rs 


exp {— abe Dare iO) 


ers) 


p(x, Bon fe) se 


eek Pe ae =| (1) 


where | /,, | is the determinant of 1, = [p,,] and p,, = 
Xa, — x, X, = Ps, 18 the correlation coefficient of x, and 
'a,. The means of x, and x, are z, and Z,, respectively. 
M,, is the cofactor of p,, in M,. : 

Let there be n zero-memory nonlinear devices (linearity 
of course being included as a special case) specified by 
the input-output relationship f;(x), 7 = 1, 2, --- , n. Let 
each x; be the single input to a corresponding f;(x), and 
designate the nth-order correlation coefficient of the 
outputs as: 


R = TT fe) 2) 


where the bar denotes the average taken over all x;. Then, 
with weak restrictions on the f;(v), we have the following 
theorem for the partial derivatives of R with respect to 
the input correlation coefficients: 


* Manuscript received by the PGIT, January 3, 1958. The 
research in this paper was supported jointly by the Army, Navy, 
and Air Force under contract with Mass. Inst. Tech. 

+ Lincoln Lab., M.I.T., Lexington, Mass. 
DH. Cramer, “Mathematical Methods of Statistics,’ Princeton 
University Press, Princeton, N. J., sec. 24.2; 1946. 


N 
Z kmormem | ay 
=1 > 


n (z 


aR 1\4 OO i) 
N a ) I] if «) (3) 
Tit(oc ee i 


m=1 


where. 7, and Sy) t= le , N, are integers lying 
between | and n, inclusive, and are me neseaears distinct. 
The k,, are positive integers, with k = peg rectal oS 
the number of times 7 appears in (7,,, S,,). 5,,s, 18 the 
Kronecker 6 function, 6,,,... = lforr,, = s,, Oforr,, ~ 8,: 
The symbol f;‘?(z,) denotes the gth derivative of f;(x), 
taken at 2;. 

Furthermore, not only is the above theorem true for 
inputs having an nth-order joint Gaussian distribution, 
but it holds true only for such inputs if the f;(x) are 
allowed to be of general form. 


Proof 


We now prove that in order for (8) to be satisfied it is 
both sufficient and necessary that the x; have the joint 
probability density given by (1). Assume that each f,(x) 
can be represented by the sum of two Laplace transforms,” 


1 ' 1 
: i se eS ' ; pi ua } — 1am 1UL 4 
f (a) On] ils h; (we du + oa ie h;-(we’™ du (4) 
where 
co ) 
hos / oer 0 
: (5) 
0 . 
hie ih EOE Vax 


and the C;, and C;— are appropriate contours. Without 
assuming any particular form for p(2, 2, -++ , %) for the 
present, 


+ 0 + co 
fee lk. 


5 Oe ad comes 


Substituting (4) in (6) and inverting the order of inte- 
gration, following Rice’s characteristic function method,” 


2D. V. Widder, ‘‘The Laplace Transform,” Princeton University 
Press, Princeton, N. J. ., ch. 6; 1946. 

38. 0. Rice, “Mathematical analysis of random noise,” Bell 
Sys. Tech. J., vol. 23, pp. 282-332, July, 1944; and vol. 24, pp. 
46-156; January, 1945. See sec. 4.8. 
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R=- 


I f ; ; ore 
(2ar9)" > ie jane 


. I] hs (e,) Olu; , Ua, = * 
i=l 


Wh. 


Ue) Qing we (7) 


where >>’ denotes a summation over all possible + 


combinations and 6(w,, Ua, , Un) 1s the nth-order 
characteristic function: 


+ oO +c +0 
Un) =s) / SEE / P(t, Le, °*° 


“exp (; De us) dx, dt, +++ dtp, (8) 
i=1 


Au, b] U2, ee ) a9) 


with j = V—1. 

We find a necessary condition for (3) to be satisfied by 
setting N = 1 = k = k,. The partial derivative of the 
left-hand side of (3) is taken on 6 in the integrand of (7), 
and the derivatives of the right-hand side are taken using 
(4). Thus the necessary condition: 


SE? if | ae ' haan) oO fey 
Cit JC. Cn*® i=1 Opie, 


Oryey 
a (5) CEP OF Gales @ sug} du, du, --+ du, = 0 (9) 


, Un) 


is obtained. The term in braces must be zero in order to 
satisfy (9) for arbitrary f;(7) and hence h;.(u). Integrating 
the resulting equation for all (7, s,) (but taking into 
account that p,, = p.;), 


5s) 


i = 
= Eps 


r=1 8 


log @(u,, U2, °°° 
PrsllUs + Gy, U2, °° * 5 Un) (10) 
=1 
where g is some function which must now be found. 
Let p,, = 1 for all (r, s). Then all the xz; are completely 
correlated, and p(2,, %2, --- , %,) can be written: 
P(1, Z2, >> + , Zp) = plz) I] A(z, — 2 +2, — 2) (1) 
where 6(x) is the Dirac 6 function. Substituting (11) in 
(8), @ is of the form: 


Hat tay ok neo, == OX (; »> weal DS us), 
t=1 t=1 
fOr allege =a) (12) 
where 
gi (u) = i pri(xy = Te GAGs = i). (13) 


Similarhyetwhen. py. °=— LL rip. eo. =  oreall 
fimo ondepes = tor all yror Ss: ls  thentro ee 
are completely correlated with (— 2,) and we obtain: 


Wt) = EX (; ¥ weal — Su), 

i=1 1=2 
—1forallr # 1, 
(14) 


5 Xn 


A(u,, U2) Yh 


for pi = 1, pir = pn = 


S00” pi. — 1b Alor all cr, Sek. 


June 


Substituting (12) in (10), we find: 


+) Ua eeeag SS use, + o( us) (15) 


t=1 


guy » U2, * 


where g2(u) = log gi(u) + wu’/2. On the other hand, 
substituting (14) in (10) yields 


gu, Us eo 


,Un) = 9 Due; + o(2u, - Fu). (16) 
t=1 1=1 

Since uw, and >>"_, u; may be considered as independent 

variables, the only solution which renders (15) and (16) 

compatible is g.(w) = K, a constant. Thus, finally, we 

have from (10) and (15) the necessary condition: 


Alea) 


= EXP [2 SF Dy Py sUpUs + 7 » Uae, + x|. (17) 


n n 
=l1 s= 


Au, , U2, Yee 


This is recognized to be the characteristic function of the 
n-dimensional Gaussian distribution* of (1) (K = 0 for 
proper normalization). 

It is now a simple matter to prove the sufficiency of 
(17), and hence (1), for satisfying (8). Using (17) in (7), 
and remembering that p,, = p.-, 


(]1) dh |e 
TI (8peme,)" () 


m=1 


n 


m 
. Uy 
i=1 


iMez 


hy x(ae,) Oty fig stt,) day duzy =e due (AS) 


By analogy to (6) and (7), and differentiating (4) with 
respect to x, the right side of (18) is seen to be equal to 


+ 00 
ile 


N 
(2 tits) 
, Xn) dx, dt, -- 


be 


thus yielding (3). 


(@,)\p Gr, ta da, 9} 


A Spectral CASE AND Its APPLICATIONS 


Consider the familiar situation where n = 2, and let 
p denote the crosscorrelation coefficient of x, and 2». 
Then (3) yields 


RA tee 
ap. = PONE GA 


(20) 


Suppose that 2, and 2, are values of a stationary Gaussian 
time series x(t) whose autocorrelation function is p(r). 
2, 18 taken at time ¢ and x, at time (¢ + 7). R(r) will 
denote the crosscorrelation function between the outputs 


4 Cramér, op. cit., sec. 24.1. 
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of two zero-memory nonlinear devices whose inputs are 
x(t) and x(t + 7), respectively. 

Taking the particular case where x(t) = 0, «°(t) = 1, 
and using (1),° 
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We find easily 


a R(1) — FCPS Ck) aS ae PACE) 
it = OME F a= ff 


Kq. (21) is particularly simple when the f;(x) are 
piecewise-polynomial functions and k is sufficiently high. 
Then the f‘"’(x) consist entirely of 6 functions of various 
orders and the integral can be easily evaluated. 

It is often of interest to obtain the derivatives of a 
crosscorrelation function with respect to r. It is con- 
venient to break down such 7 derivatives into a series of 
products of derivatives of R(r) with respect to p(r), and 
p(7) with respect to 7, using 

dR(r) | AR(t) dp(r) 


dr . Op(r) dr 2 


This enables the nonlinear devices to be treated in- 
dependently of the shape of the input correlation function 
p(r), using (21). Similarly, the derivatives of p(7) with 
respect to 7 do not involve the f;(z). 

As an example, Cohen" shows that in general, for 
autocorrelation functions R(r), the limiting behavior of 
the corresponding power spectrum ®(w) is given by: 


| lim w?&(e) = —+ (BO) 
@— co Tw ca [7=0+ (23) 
3 
lim o'| oat) ee | ee 
us dr 7=O+ us dr 7=0+ 


| @> 0 


‘and so on, where the derivatives are with respect to r. 
Another application of (20) is in deriving Bussgang’s 
‘interesting result’ that the crosscorrelation function 
\between the input and the output of a nonlinear device 
driven by Gaussian noise has the same shape as the 
input autocorrelation function. In this case f;(x) = «x 
and f,(a) is arbitrary. Then f/(x) is unity, and all higher 
derivatives of f,(a) are zero. Putting this into (21) and 
evaluating the integral, 


“© (x) exp (—2°/2) At 
se) _ | f Dds k= 1 oy 
dp(7)* | 
LO be 1 


5T. N. Amiantov and V. I. Tikhonov, “The effect of normal 
uctuations on typical nonlinear elements,” Bull. Acad. Sci. USSR, 
p. 33-42; April, 1956. Here, an autocorrelation case [fi(x) = fo(x)] is 
tudied by expanding the second-order joint Gaussian probability den- 
sity of x(t) and x(t + 7) in powers of p(r), using Mehler’s formula; see 
ateman Manuscript Project, ‘Higher Transcendental Functions,” 
McGraw-Hill Book Co., Inc., New York, N. Y., vol. 2, p. 194, (22); 

953. They then integrate by parts to obtain the output correlation 
function in a similar series but do not recognize the simple form (21) 
or this series. Using this method it is not required that fi(x) be 
aplace transformable, rather than our proof. } 

6 R. Cohen, “Some Analytical and Practical Aspects of Wiener’s 
heory of Prediction,’ Res. Lab. of Electronics, M.I.T., Cambridge, 
ass., Tech. Rep. No. 69, ch. 4, sec. 2; June 2, 1948. ; 
7J. J. Bussgang, ‘“Crosscorrelation Functions of Amplitude- 
Distorted Gaussian Signals,’ Res. Lab. of Electronics, M.I.T., 
ambridge, Mass., Tech. Rep. 216, sec. 3; March 26, 1952. 


wer AC) tas Oa 
RO) = or) | A ay 25 
ae FOE (25) 
exp {—[zi + a — 2p(r)ax2]/2[1 — p"(7)]} : 
> shh d. 196 21 
De ie p (7) pee ey) 
thus yielding Bussgang’s result. Unlike Bussgang’s 


theorem, (20) cannot be generalized to hold for probability 
distributions other than Gaussian.*~”° 


Some Simpte AvTOCORRELATION EXxAmpLes [For 
a(t) = 0, x(t) = 1 


Hard Limiter 
Van Vieck’s well-known result on the autocorrelation 
function of the output of a hard limiter’’ can be derived 
very simply, using (21). If 
ie ee 
see (26) 


fix) = f(x) = 
Tan (() 


—{; 


then f{’ (x) and f§' (x) are first-order 6 functions of area 


2,atz = 0. 
Substituting in (21) and integrating, 
OR(7) a 2 ete (27) 
dp(z) rV1l— p (7) 
When p(r) = 0, R(7) = 0. Thus 
2 iva dp(7) Be a) 
R= 2 | Ras = Zain OT 8) 


which is Van Vleck’s result. 


Linear Detector 


Similarly, the autocorrelation function of the output 
of a linear detector can be easily found. If 


ap > (0) 


a) 


AORN Koel 


Lo; 


then f{” (x) and fs” (x) are first-order 6 functions of area 
unity at x = 0. Substituting in (21) and integrating: 


(29) 


a°R(r) te if ; 
OpG) "2547 pe) 


(30) 


Doubly-integrating (30) with the boundary conditions: 


8 J. F. Barrett and D. G. Lampard, ‘An expansion for some 
second-order probability distributions and its application to noise 
problems,” IRE Trans. on INrorMaTiION THroRY, yol. IT-I, 
pp. 10-15; March, 1955. 

9 J. L. Brown, Jr., “On a cross-correlation property for stationary 
random processes,’ IRE Trans. oN INFORMATION THEORY, vol. 
IT-3, pp. 28-31; March, 1957. 

10 A. H. Nuttall, “Invariance of Correlation Functions under 
Nonlinear Transformations,’’ Res. Lab. of Electronics, M.I.T., 
Cambridge, Mass., Quart. Progress Rep., p. 63; October 15, 1957. 

uJ, L. Lawson and G. HE. Uhlenbeck, “Threshold Signals,” 
McGraw-Hill Book Co., Inc., New York, N. Y., p. 58; 1950. 
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IER ( +) Peace} Sigg Ge - 
ee — | fy. (2) - a ae | = ‘ 
# ck for a0 Ol 
pe —22/2 2 1 
R ie | Xe ar | vod 
(17) | f vee om 
we obtain: 
Sel 5 dt | 
R(r) = | E Es | we ee 
d Jo 0 Oe rie _— - } 2a 
l f / aa ee eee cet) ee oo ‘ 
= Dy \ PA?) cos [p(7)] + Vie p'(r) (32) 


. st ° . 12 
which is in agreement with Rice’s result. 


Clipper 

The relations derived independently by Robin’* and 
Laning and Battin’* for the autocorrelation function of 
the output of a clipper may also be found by this method. 
With a clipper characteristic: 


<a 


fix) = fr(x) = | x; Sea al (33) 
—l 2 


a<l 


and fi? (a2) and fs” (x) each are a pair of first-order 6 
functions at x —l and x = l, with areas 1 and —1, 
respectively. Substituting in (21) and integrating, 


ie iP 
“olepicele ts rae 
rV 1 — p(7) 


result, for input noise of unit variance. 


I Rr) = 
dp(r)” 


which is Robin’s 


(34) 


Smooth Limiter 


a : : 15 . 
Finally, Baum’s recent interesting result” for the 


12 Rice, op. cit., eq. (4.7-5). 

Beals Robin, “The autocorrelation function and power spectrum 
of clipped thermal noise. F itering of simple periodic signals in this 
noise,’ Ann. Telecomm., vol. 7, pp. 375-3887; September, 1952. 

14 J. H. Laning, Jr. and R. H. Battin, “Random Processes in 
Automatic Control,” McGraw-Hill Book Co., Inc., New York, 
N. Y., p. 362, eq. (B-8); 1956. 

6 R. 1S Baum, “The correlation function of smoothly limited 
Gaussian noise,’ IRE Trans. on INrorMaTION TuHeEory, vol. 
IT-3, pp. 193-197; September, 1957. 


INFORMATION THEORY June 


autocorrelation function of the output of a device having 
an error-function characteristic will be derived. With 


] Yo eayons 
fila) = fo(x) = a | e dt (35) 
we have 
Pa y= =A ea Can ae oye 2a / 22 (36) 
Substituting in (21): 
a 
dp(7) “2r' NL ip @) 
iki + pits — 2pox12%s 
ia fre Ae 201 ca 2) Ie dat» (37) 
rN pi = ps 
where 
{PD =O) eo] 
pli Ie ae! (38) 
5 ae oo = OD) 
= AUT pa) fetal eat) 


The term in braces in (37) must equal unity, since it is 
the integral of a second-order Gaussian probability 
density. Thus, from (38), 


aR) 1 ip ta 
dp(r) 2x NI — p(z) 
eV + 1)? = 1 *p%(a) 
Integrating and using the condition that when p(r) = 
R(r) = 0, 
i hog tea lay dp(7) 
R(t) = oa 5 
2 Ginn wsVaOte Ltrs) 
be idea -1 elon 
“5 sin E ce =| (40) 


which is in agreement with Baum’s result. 


a 
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The Effect of Instantaneous Nonlinear Devices 
on Cross-Correlation® 


ROY LEIPNIKT 


Summary—lIf X,(t), X2(t) are two noises (stochastic processes), 
f and g are functions describing the action of two instantaneous 
nonlinear devices, we say that the (m, n) cross-correlation property- 
holds in case the cross-correlation of f(Xi(ti:)) with g(X2(f.)) is 
proportional to the cross-correlation of Xi(t2) with X2(t.), whenever 
f and g are polynomials of degrees not exceeding m and n, re- 
spectively. We take m =~ or n = to mean that f or g is any 
continuous function. 

The Barrett-Lampard expansion? of the second-order joint density 
of Xi(t:) and X2(t.) is used to derive an expression for the cross- 
correlation of f(X,(t,)) and g( X2(t2)). This expression yields neces- 
Sary and sufficient conditions for the validity of the cross-correlation 
property in three cases: X,(t) and X,(¢) stationarv, m, n unrestricted; 
X,(t) stationary, m, n unrestricted; X(t) stationary, n = 1. 

Examples are constructed with the help of special orthonormal 
polynomials illustrating the necessity and sufficiency of the con- 
ditions. 


) INTRODUCTION 

) USSGANG' showed that if one of a pair of 
B stationary Gaussian processes is amplitude-dis- 
| torted in an instantaneous nonlinear device, that 
the cross-correlations before and after distortion are 
proportional. In our terminology, this is the (, 1) 
cross-correlation property. 

_ Barrett and Lampard’ introduced a_bi-orthonormal 
expansion method in a successful attempt to extend 
‘Bussgang’s result to nonstationary processes with joint 
distribution whose bi-orthonormal coefficient matrix 
iis diagonal. The latter is sufficient, but not necessary for 
‘the (©, 1) cross-correlation property. Brown* found a 
mecessary and sufficient condition for the (@, 1) property 
when X,(¢) and X,.(t) are both stationary, and for a 
partially time-dependent (© ,1) property when X,(¢) 
is stationary, X,(t) nonstationary. 

It is natural to ask whether a like property holds when 
both processes are distorted by instantaneous nonlinear 
devices and whether conditions can be relaxed if one or 
both devices are of restricted complexity (such as quadratic 
detectors). 

We find that the results of Brown can be fully general- 
Red in both of these directions (Theorems 1 and 2). We 
Further discover (Theorem 3) that the partial time 


* Manuscript received by the PGIT, September 16, 1957; 
evised manuscript received, February 25, 1958. 

+ Dept. of Mathematics, University of Washington, Seattle, 
ash. 
14. J. Bussgang, “Cross Correlation Functions of Amplitude 
Distorted Gaussan Signals,’ Res. Lab. of Electronics, M.I.T., 
Jambridge, Mass., Tech. Rep. No. 216; March 26, 1952. 

2J. F. Barrett and D. G. Lampard, ‘‘An expansion for some 
econd order probability distributions and its application to noise 
roblems,’”’ IRE Trans. ON INForMATION THEORY, vol. IT-1, 
p. 10-15; March, 1955. 

3 J. L. Brown, Jr., “‘On a cross-correlation property for stationary 
‘andom processes,” IRE Trans. oN INFORMATION THEORY, vol. 
T-3, pp. 28-31; March, 1957. 


dependence allowed by Brown in the proportionality 
constant is absent. when a stationary process is distorted 
and a nonstationary process is undistorted. 


ANALYSIS* 


Let pi, 1,(%1, %2) be the joint density of two stochastic 
processes X,(t) and X,(t,). If both are stationary, the 
time dependence involves t, — t, only. With Barrett 
and Lampard’ we introduce the marginal densities 
Pi(X:), Pr(X2), Sequences {6;'1,(%1)}, 1::1,@2)} of poly- 
nomials orthonormal with respect to p,(a,), p2(a,), and 
the matrices A,,,., = [@n.2::,,1,| such that 


Disc Gay) 
Pr (a) pr (£2) 


has the formal bi-orthonormal expansion 


ye Amn, tr, te Os. ay) OS (a5) 


We say the (m, n) cross-correlation property holds if 
for each polynomial f of degree < m, each polynomial g 
of degree < n, there is a constant k(f, g) such that 
Pr. .ts(f, 9) = K(f, 9) Pe... Where 


COV (f(Xi(h)), g(X o(te))) 
Vvar (f(X,(4)) -Var (g X2(t2))) 


COV (X,(4), YAS) : 
V (war Xi(t,)) (var Xo(t,)) 


Paes g) 5 


Pitas (1) 
The conditions derived below depend on a lemma stated 
here and proved in the Appendix. 
Lemma: If 


Cian = | On.) (ty) f(a) py, 1,(@1) da, 


d.it, = ‘| 65; (2) Glas) Do (2a) dx, 


for? aise ORI 2. socom 


where f and g are polynomials of degrees S m and S n 


respectively, then 


n 


M 
Dy SS Colle hase eee 


u=1 v=1 - (2) 


m 7 n 
2 2 
cae dae 


u=1 v=1 


Pisces q) = 


4 Questions of convergence are avoided in the formal calculations 
here. A rigorous treatment of the Barrett-Lampard expansion may 
be found in a forthcoming paper by the author in the J. SIAM. 
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‘This simple bilinear expression immediately yields 


Theorem 1 

if X,(t), X.(t) are stationary, then the (m, n) cross- 
correlation property holds if and only, if there is a time 
independent m X n matrix Q = [q.oln» With gq. = 1 
such that the upper left m X n submatrix B,,., = 
[Ag. iclm Of A:r,2, Satisiies the equation B,,, = 


Qi 41,t, ta- 
Proof: Since X,(¢) and X,(¢) are stationary, the Fourier 


coefficients of f and g are time-independent: c,,,, = 
C,; Os.2, = d, torte = Lies nyu Looeet ne Hence 
Sy Ss Cu dA, V,t1,to 
u= v=1 
rte ths g) aa my 
I? 
rs Sa 
v=1 


Note that p.,.., = If the matrix equation holds, 


then 


Cinta aba. 


3 LG dy Qus 
Dae Da 


a K(f, 9) Pts, ts 


(Actually, the correlations depend on ¢, — ¢, only). 

Conversely, 1f, pin, 0).0) = MG Oe Puce BOP DOly- 
nomials f, g of degrees S m,n, pick f, = 0), ¢. = 0, 
sorthat ¢, = 0,2, ds = OvsnsPiy ein Or) = Cian, = 
K(f 793) G4 1, 6.. Lauspwe haved. -, =) ccs, 2g,)=20r 
ioe | eee ey 1, i aNd Bye, == 10s ee. 

If only one of the processes is stationary, results are 
still obtainable. 


veces g) = = 


Pix.ts 


Theorem 2 


if X,(@)-is.stationary, then p;, 1.0, 9) = A..G, 9) p:..i, 
for all polynomials f, g of degrees S m, n if and only if 
there exists an m X n matrix Q,, such that B,,,, = 


Qy 41,0, ,t2Qe,- 
Proof: We have ¢,,:, = ¢, foru = 1, - 


> Dae as toAu, v,t1,ta 


Pree Ulban taet — aes 
dC Cy > ws, ts 


If the matrix equation holds, then 

= ae dag Vinties 

fs Se Da, 
v=1 


i NEA 9) Ptx,te 


- m, so that 


Bren. g) = Ptsts 


The converse is proved as above. 
The time dependence of the proportionality constant 
in Theorem 2 above drops out, yielding the cross-corre- 


June 


lation property of primary interest, in case it is only the 
stationary process X, which is nonlinearly distorted. 


Theorem 3 


If X,(¢) is stationary, then the (m, 1) cross-correlation 
property holds if and only if there is a m-vector q = [q,]m 
with g, = 1 such that the m X 1 upper left vector b,, .., 
of Ay, «4, Satishes O35), =O; eG: 

ee Since X,(é) is stationary, c,,,, = ¢, for wu = 1; 

. Since X,(¢) is passed through at worst a linear 
sevice Oniite Opes DLS 


DEH ins sare 
Piaf 9) = == 


Pars 


u=l1 


If the vector equation holds, then 


m 


mec 


Poten Is 9) = aierses Pty, to: 
a] UG 
u=1 


The converse follows as in Theorem 1. 
The results of Brown follow from Theorems 1 and 2 on 
setting m = o,n = 1. 
EXAMPLES 


1) The first example considered is where X,(t,) and 
X(t.) have a joint Gaussian density 


1 
Dir ,te(%1, X2) = ()_() 2 
Qro{Po?V1 = pir, 
y 
1 Ee Th BD 3 
“exp 4 — 25.02 EG) aay oe 
2(1 or eae te) te el UE ee 


with marginal densities 


1 72) a 
— —_—_—_ —— ss 1 
Pus) = es ox { oo 


ty 


1 1/2 x 
P1,(X2) Fi (=) exp ae 


to 


The orthonormal polynomials are Hermite polynomials: 


Oe) = Gm | aR) 
6° (2) = Cae H,(- (2) oT 


We then have 


D1,(%1) Pe ,(X2) ae A een 


1 2 
*€xp iS Teagan Ex 
1 Pts, te 


Dae =. on 
—~ 20.51 2o' wrg® 4,5 |} = DD: Ptvct On (8) On” eae 
n=0 


Ot, 
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Hence 4, .1,,u,0 = P:,.1.9.,2. Lt follows that if X,(é,) and 
X(t.) are uncorrelated (9,,,., = 0), then so are f(X,(t:)) 
and g(X2(t.)) for all choices of f and g. 

Choose f(x) = g(x) = 2’, so that 
~=0,;, G1, = Vee 
Ber) ee ee een 


We thus have 


O} 


Puen ts 9) = DEM Gi! Pia ds 
If X,(¢,) is stationary, then 


(2)? 2 


Piegecai, g) = Phe ane Dinies 
If also X,(t,) is stationary, then 


Pewee g) = ena Tp oat. 


The 2 X 2 matrix of a’s is 


0 Aaee 


Clearly the matrix equations and (2,2) cross-correlation 
‘conditions both reduce to p,,,,, = k and p.,,., = k,, in 
the cases taken up in Theorems 1 and 2. It follows that 
if the cross-correlation is nonconstant in the doubly 
stationary case, the (m, n) property fails if m = 2 and 
in = 2. 

2) The second example, or family of examples, is 
constructed with the help of Legendre polynomials. We 
find distributions satisfying: a) the (m, n) property but 
mot the (m, n + 1) or (m + 1, n) properties, and b) the 
((@, ©) property. 


lt must be admitted that these examples are rather 
artificial. 

Let {s,(a)} be the polynomials orthonormal on [0, 1] 
with respect to the rectangular distribution (unit weight 
function). Thus” 


(2k)! 
Pa ae 


s(x) = V2k +1 P,(2¢ — 1), 

twhere the P,(a) are Legendre polynomials. 

‘Set T,(y) = f% s,(@)dx. Since s(x) = 1, we have 
fy) = ¥. 

Clearly 7,(0) = O for all &. Moreover, T,(1) = Jo 
y(x)dx = Jo 8 (x)So(x)daz = 6,0, so that T,(1) = 0 for 
= 1. Since s,(a) is a polynomial, 7,(y) is a polynomial, 
and we can write T,(y) = y(l — y) U;(y), where U,(y) 
is a polynomial of degree k — 1 for k 2 1. 

| Now set 


(t1, %) = bs b,,:T'(a1)T (ao), 


4,1=0 


where bo.o = if b;.0 = bo, == 0, 12 1, 


nd (2%, #2) is in the unit square. 
5 Follows from the normalization found in A. Erdelyi, “Higher 


‘ranscendental Functions,’ McGraw-Hill Book Co., Inc., New 
Work, N. Y., vol. 2, p. 179; 1953. 
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We wish to find conditions for F(a, x2) to be a joint 
cumulative distribution on the unit square. Note that 
F(a, 0) = F(O, x2) = 0, and that 
Fay |) = 2D Bol (ay) == Ly) = am FC, 3) = a. 
Thus the marginal cumulative distributions f,(2,) 
and F(z.) are rectangular, with densities f,(7,) = 
f.(@2) = 1. It remains only to show that F(a, x2) is jointly 
monotone in both variables on the unit square. If the 
joint density f(v,, x2) = 0°F/0,,0,, exists in the interior 
of the unit square, the monotone character of F(a, x2) 
is insured by the nonnegativity of f(#,, v2). Observe that 


fei, x2) . oe 
f(2)folas) f(a, %2) = 1+ = bj ;8:(a1)8;(a2)- 

Because of the alternating character of Legendre poly- 
nomials, not every choice of (b,;;) will make f(a, z2.) 2 0, 
so that a certain discretion must be exercised. However, 
a wide latitude is available. The classical inequality® 
| P,(z) | Ss 1 for — 1 Ss 2S shows that its @ aio 
/2k + 1 (2k)!/2*-(k!)? fork = 0,0 < x < 1. Hence 
it is sufficient to so choose the b,,; that 


HI)! 4 
ayy = 


for all choices of sign. This crude condition suffices for 
the construction of the first example. 


1+ S 4 bu VQi+ DAI + D- 


t,7=1 


a) Let 
bi 1 a AR re ay UG ae 
and let 
Du venat = Bi, t2 = Ate—-ty 


where p;,—,, and a,,-,, are functions such that 


Pto—ts 


Ota 


is nonconstant and 


Ince Spin ohne Cree) Crees) 


: (2m + 2)'(2n + 2)! 
(m+ Din + 1))?2"*"*? 


is nonnegative for all four choices of sign. We set all the 
other b;; = 0 for 7, 7 2 1. It follows from Theorem 1 
that the (m, n) cross-correlation property holds, while 
the (m + 1, n) and (m, n + 1) properties fail. 

The other type of example, in the opposite direction, 
is more elegant. 


b) We begin with the little-known identity’ 


— 2k + 1 
ED 


— log [(1 — min (a, y))(1 + max (a, y))] 


P,(z)Pily) = —1 + 2 log 2 


§ Tord., p. 205. 
7 Ibid, p. 183. 
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valid for (x, y) in [— 1, 
square, this becomes, 


1] X [— 1, 1]. Shifted to the unit 
after some manipulation, 


oo 2?*(k!)* a 
2 ik DIGI? see) = —1 — log Glas, #2); 


where G is the Green’s kernel 


. eget ae 
G(x, %) = Exe X2) 1 = t 
exe a eiNG Ly Ss Bo 
Since 0 S G(m, x.) S 1 in the unit square, — log 
Gorrie) =.0: 
Now choose as coefficients 
Dekee a! 4 ; 
Dee ay LOR ed cee! 


Pts—ts (a+ 1)((2i)!)? 


Thus we have 


f(a, 2) 


CAINE Te ee Ee YS buss.Can)s, (cs) 


4,7=1 


SM Direct 2p ele Oe Gti es) | 


If the cross-correlation p,,-,, satisfies the inequalities 
0S p:.-:, S 3 for allt, — t, then f(@, 2.) 2 0, and we 
have a legitimate distribution. (If p,,_,, = 0, the example 
reduces to independent rectangular distributions). Since 
the conditions of Theorem | are satisfied for all m and 
all n, the above joint distribution has the (~, ©) cross- 
correlation property. In other words, each of two pro- 
cesses with the above joint distribution can be distorted 
through arbitrary instantaneous nonlinear devices without 
altering the time dependence of the cross correlation. 


APPENDIX 


We now derive the Lemma. Let 


sine =f OP CMIDL ult) de, dy 


=i) 6,71 .(t2) G(t2)P2.1,(2) Ax», 


where f, g are polynomials of degrees m, n. Since 65’) («) = 
6;7)(x) = 1, we have 
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Conta ae [ t@n..@) da, = E[f(Xi(t))], dor, 


‘a f G\ a2) D2, 1;(%a) dx, ae El g(Xo(t))]. 


Hence 
fla) — BUf(X,(t))] 
= Dy 01 60a g(t2) — E[g(X.(t))] 
= 3 dy.,6°? (a), 
and as 
B((Xi(t)) — ELK (4)))G(Xelt)) — Elg(X2(t))))] 


(23) dz 8 0.7) (ae) dx, diy 


= i Pis.ta(Br, La) Ss pare 
u=1 v=1 
dove. ‘it » Gs i ,t1.t2Dt,(L1) Pe, (Lo) 


Ds 2 Cu, ty 
OS) (x1) 057), (ato) Oy. (1) 0, +. (Bo) day cam 


I 


u=1 


m 


n 
SS, DS Ss, Cu, ts Gh, a,@s. ss tate Onion 
459 


u=1 v=1 


m n 
2 Se cers Os Oa hee mee 


u=1 v=1 


on using the biorthonormal expansion for p,,,., (1, 22) 
and taking account of the orthonormality of {0°, (a,)} 
and {6° (v.)}, Similarly 


HUG (4) = Elf(X(4))/)7] 
= [ mateo 3 o....00%, Ce de 


™. n 

ee tion: [oda 
u=1 v=i1 

™ 

Cee 

u=1 


(1) 


O,'1.(21) 05.) (a) dae 


I 


and 


Elg(X2(t.)) — Elg(X2(t)))"] > det. 


from which the desired result follows. 


Za 


958 
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Some Properties of Nonbinary Error-Correcting Codes’ 


CAN a cE Enh 


Summary—An error-correcting code may be thought of as a 
ubset So of points belonging to a set S in which a metric is defined 
uch that the distance between every pair of distinct points of Sp 
s larger than some given number. In Hamming’s original formu- 
ation, S was taken to be the set of all 2” n-bit binary numbers 
nd the distance between a pair of binary numbers s and f was 
aken to be the number of bits of s which do not agree with the 
orresponding bits of f. In this note we shall take S to be the set of 
Il n-tuples in which each coordinate of an n-tuple can assume 
ne of k integral values: 0, 1, ... , R — 1, with k = 2. Properties 
f these nonbinary codes will be discussed. 


N error-correcting code may be thought of as a 
A subset S, of points belonging to a set S in which 

a metric is defined such that the distance between 
very pair of distinct points of S, is larger than some 
iven number. In Hamming’s original formulation, S 
yas taken to be the set of all 2” n-bit binary numbers 
nd the distance between a pair of binary numbers s and 
was taken to be the number of bits of s which do not 
gree with the corresponding bits of ¢t. In this note we 
hall take S to be the set of all n-tuples in which each 
pordinate of an n-tuple can assume one of k integral 
alues: 0, 1, ---, k — 1, with k = 2. These k values have 
ne usual ordering and addition modulo k is assumed 
efined. For instance, if k = 7, then 4 + 6 = 3 and 
_— 5 = 6 (mod 7). In our case a metric can rather 
hturally be assigned to S and a result of Hamming" can be 
ctended directly to group codes when each of the co- 
tdinates assumes one of a prime number of values 
|.e., k is prime). This extension is implied, but not 
oved, in a recent paper by Ulrich.” A method is also 
ven for constructing certain single and double error- 
brrecting group codes for arbitrary k. lor nongroup 
»des, we extend several inequalities on code size obtained 
r binary codes by Plotkin.*® Finally, we obtain all close- 
acked double error correcting codes for k = 3. 
/From here on, an n-tuple is called a word of length n 
nd each coordinate of an n-tuple (word) is called a letter. 
he set of all words of length n in which each letter can 
sume one of k& integral values 0, 1, ---, k — 1 is desig- 
ited by S,(n). S,(n) has k® members. Any subset of 
.(n) is called a code. 
‘Consider the k values of each letter as points placed 
ong a circle such that the circle is divided into k arcs 
equal length. Let the distance between two letters 
land y be the smallest number of arcs separating x and 


|* Manuscript received by the PGIT, January 6, 1958. 

+ Bell Telephone Labs., Inc., Whippany, N. J. ! 

1R. W. Hamming, “Error detecting and error-correcting codes,”’ 
ll. Sys. Tech. J., vol. 29, pp. 147-160; January, 1950. 

|2W. Ulrich, “Non-binary error correction codes,” Bell Sys. 
ech. J., vol. 36, pp. 1341-1388; November, 1957. 

3M. Plotkin, “Binary Codes with Specified Minimum Distance,” 
pore School of Elec. Eng., University of Pennsylvania, Phila- 
\lphia, Pa.; 1952. 


y. Let the distance between two words be the sum of the 
distances between their letters. Such a distance function 
is called a (circular) metric for S,(n). More precisely, if 
§ = (8), 8, °°" , &) and t = (jt, >= 45 4,) ate two words 
of S,(n), then p is the circular metric if 


n 


p(s, t) = om p(S;, t;) 


4=1 
where 


p(s, t;) = Min {s; Sa t;, t; T= 8;} (mod k). 


The characteristics of a circular metric are exhibited by 
certain physical devices such as circular print wheels or 
ring counters. 

As illustration let us observe several examples of 
ternary (k = 3) codes where each letter can take on one 
of three values 0, 1 and 2. The code C consisting of the 
following three 3-letter words in S = §S;(3) 


000 
1 ots PS 
222 


is a ternary single error-correcting code. Should one of 
the words, say 111, be sent and a wrong word, say 121, 
be received, an encoding device could be instrumented 
to correct this single error since 121 is of a distance at 
least two away from the other words of the code. 

We note that each member c of the code C has exactly 
six neighboring words in S with distance one away. These 
six words together with c form a closed ball of radius 
one about c. Since there can be not more than 3°/(6 + 1), 
or less than 4 isolated balls of radius one in S, it follows 
that there can be not more than three words in any 
3-letter ternary single error-correcting code. Thus we 
say the code C is full in S or the biggest possible in S. 

We also note that S is an abelian group under addition 
modulo k and that the code C is a subgroup of S. C is 
therefore said to be a group code in S. 

Following this line of thought we observe that the code 
D consisting of the following nine words in the set 
T = 8;(4) of all 4-letter ternary words is again a single 
error-correcting code: 


Let Tae SSS Sr ts) (= 
[SINS ISS SSS 
SS) See is 
RS RSS iS iS SS =) 
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Since 3°/(8 + 1) is exactly 9, it follows that no 4-letter 
single error-correcting code can have more than nine 
members. Also, D is a subgroup of 7 under addition 
modulo 3. Therefore D is a full group code. 

There is an additional property of the code D which 
is not shared by the code C. Each member d of D belongs 
to a ball of radius 1 about d consisting of nine members 
of 7. If we take the union of all these balls over all members 
of D, we find that this union is exactly T and not smaller. 
We say the code D is close-packed in T. 


I. THEOREMS 


We shall state and discuss several results which will be 
proved in Section ITI. 

As given previously, let S,(m) be the set of all words 
of length nm in which each letter can assume one of k 
integral values 0, 1, --- , k — 1. A subset S, of S,(m) is 
said to be a single error-correcting code if the distance 
between every pair of distinct members of So is 3 or 
greater, and in general, a subset S, of S,(m) is said to be 
an e-error-correcting code if the distance between every pair 
of distinct members of S, is 2e + 1 or greater. For given k 
and n, we shall let a,(n, d) denote the size (the number of 
elements) of the largest e-error-correcting code in S,(7) 
and we shall let b,(n, d) denote the size of the largest 
e-error-correcting group code in S,(n) where d = 2e + 1. 
Then 


Theorem | 


Let k be an odd prime. If 2n + 1 is not a power of k, 
then 


b,(n + 1,3) = kb,(n, 3). 


If 2n + 1 isa power of k, then any group code with 6,(n, 3) 
members is close-packed, and 


b(n + 1,3) = b(n, 8). 


For k = 2, this is essentially Hamming’s result.* 

Single error-correcting group codes in which k is prime 
are completely characterized by Theorem 1. In the case 
k is not prime, Theorem 1 fails but we have 


Theorem 2 

Let m be some given positive integer and let C denote 
a single error-correcting group code. 

1) For all odd k, k > 2, there is an (m + r)-letter code 
C such that C is close-packed and C has k" members 
where r = 3(k” — 2m — 1). 

2) For k even, k > 2, there is an (m + r)-letter code C 
with k’/2 members where r = 3(k™ — 2”). 

Theorems 1 and 2 indicate the behavior of single 
error-correcting group codes, but much less is known 
about double error-correcting codes. From (ii) of Theorem 
2, we may show 


4Tf 2n + 1 is replaced by n + 1, then the case k = 2 is exactly 
Hamming’s result. 


Jun 


~ 


Corollary 1: Let k be even, k > 2. For a given m, le 
r = 4(k" — 2”). Then an (m + 2r)-letter double error 
correcting group code with k"/2 members can be con, 
structed. 

An example of such a double error-correcting code i 
as follows. Let k = 6,m = 1. Then, r=3(k" = 2") 
By the corollary, a 5-letter double error-correcting grouy 
code having 18 members can be constructed. Such a code 
is 


00000 20220 40440 
02402 22022 42242 
04204 24424 44044 
11311 31531 51151 
13113 33333 53553 
15515 35135 55355. 


So far we have been concerned with group codes. A 
more difficult problem is to construct full codes whick 
may or may not be group codes. Plotkin gave’ severa 
interesting bounds on the number of points in nongroup 
full binary codes. Extending his proofs, we may show 
the following two theorems. 


Theorem 3° 
lor all pairs d, n such that 3d > 2n, 
a,(n, d) S 3d/(38d — 2n). 


9, which 
Zils 


Some examples of this theorem are a;(4, 3) 
we already know, a3(7, 5) S 15 and a,(10, 7) 


IA IIA 


Theorem 4 
For all k, 
a,(2n, 2d) = an, dan, 2a); 


As an example, we know a;(11, 10) = 3, by Theorem ¢ 
and from the next section, we shall see that a;(11, 5) = 3.' 
Therefore, by this theorem, we have a;(22, 10) = 3’. 

The proofs of these two theorems will be left to the 
reader. 


Il. Some Remarks ON CLOSE-PACKED CopEs 


In the binary case, Golay showed® that there is < 
close-packed, 23-letter, triple error-correcting code having 
2" members. It can be shown trivially that there is < 
close-packed, 5-letter, double error-correcting binary 
code having two members. Shapiro showed’ (unpub: 
lished) that there can be no close-packed double-erro: 
correcting, binary code* for n # 5 and n ¥ 90. Moreover 


° The following extension to arbitrary k is probably true: a,(n, d 
S< 4d/(4d — kn) when 4d > kn for k even, ai(n, d) S 4kd/(4kd — 
(kK? — 1)n) when 4kd > (k® — 1)n for k odd. 

°M. J. E. Golay, “Notes on digital coding,’ Proc. IRE, vol 
37, p. 657; June, 1949. 

7 After this paper was submitted, the author learned that thi 
unpublished material referred to will appear in a forthcoming pape 
by H. 8. Shapiro and D. E. Stotmiek, “On the mathematical theor 
of error-correcting codes,” JBM J. Res. 

*S. P. Lloyd, “Binary block coding” Bell Sys. Tech. J., vol 
36, pp. 517-535; March, 1957. It is shown here that there is n 
close-packed code for n = 90. 
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1e showed that there are at most a finite number of close- 

acked, e-error-correcting, binary codes over all n for 
» = 2. We shall add a few facts here concerning k-valued, 
‘lose-packed codes. 

It follows from Theorems | and 2 that for each odd k, 

here is an infinite number of k-valued, close-packed, 
ingle error-correcting codes over all n, and it is clear 
that there can be no nontrivial close-packed, single 
prror-correcting code for k even, k > 2. Let us consider 
next double error-correcting codes. The cases k = 3 and 
«& = 4 are special and will be looked at first. 
For k = 3, a closed ball of radius 2 about any point 
of S;(n) has exactly 2n? + 1 members. Therefore, a 
mecessary condition for a 3-valued, double error-correcting 
ode to be close-packed is that 2n” + 1 be a power of 3. 
That is, the diophantine equation 


Qn? +1 = 


pe solvable in positive integral 2 and n. By chance this 
equation has been completely solved by Nagell,’ pre- 
‘sumably in no way connected with error correcting codes. 
'The only possible solutions of this equation are 


{n=1 [n=2 [n= 11 
ren ees lens 3. 


‘The first two solutions are trivially sufficient, and each 
‘code has just one member. We therefore have 

Remark 1: The only possible,” nontrivial, 3-valued 
close-packed, double error-correcting code is when n = 11. 

For k = 4, we note that if we encode each value in 
‘binary so that the single letter 0 in the k = 4 system is 
‘encoded into the 2-letter binary word 00, 1 into 01, 2 
‘into 11 and 3 into 10, then the circular metric in the 
k = 4 system is preserved in the corresponding binary 
system. Hence, every code in S,(n) is equivalent to a 
code in S.(2n). Therefore every close-packed code in 
Ps (n) corresponds to a close-packed code in §S,(2n). 
Since from previous discussion we know that there is no 
[close- packed code in S,(2n), it follows that 
| Remark 2: There is no 4- valued, close-packed, double 
error-correcting code. 

For k > 4, a closed ball of radius 2 about any point 
of S,(n) has 2n* + 2n + 1 members. Therefore, for k 
prime, a necessary condition for a k-valued, k > 4, 
double error-correcting code to be close-packed is that 
Qn? + 2n + 1 be a power of k. In other words, the dio- 
phantine equation 


In += on + 1 = 


be solvable in positive, integral n, x and prime k. 
This equation is trivially solvable for x = 1. For x > 1, 


*°T. Nagell, “Sur |’ impossibilité de quelques équations 4 deux 
indéterminées,’’ Norsk Matematisk Forenings Skrifter, series 1, 
no. 13; 1923. 

1 A’ code satisfying the third solution has been constructed by 
Golay, op. cit. 
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it can be shown that there is an infinite number of solu- 
tions. However, this equation has at most a finite number 
of solutions for fixed k (see below), and at most a finite 
number of solutions” for fixed « = 3. 

Some known solutions to this equation are 


i; = 3 le = 119 ce = 4059 
k=5 \¢4=138 +k = 5741 
le = 2 |, == 4 i — Oe 


There is no code which satisfies the first solution. It is 
not known, although somewhat doubtful, whether there 
are codes which satisfy the other two solutions. 

Extending a proof given by Shapiro,” it can be 
shown that 

Remark 3: There can be at most a finite number of 
close-packed, e-error-correcting codes for each k, k prime, 
k = 2e + 1 ande = 2. 


III. DEFINITIONS AND PRooFs 


Let S,(n) be the set of all n-tuples (a, %, --- , a,) 
where x; takes on integral values 0, 1, --- , & — 1, for 
7 = 1,2, --- ,. Each member of S,(n) is called a k-valued 


word of length n or a word if no confusion arises. Each 
coordinate of a word is called a letter. 

Now let a metric p be assigned to S,(n). We introduce 
the term proximity PGS) of any subset S of S,(n) to be 


P(S) = Min {p(z, y)|x, ye S, x ¥ y}. 


Let d be a positive integer. Any subset of S,(2) whose 
proximity is not less than d is said to be an e-error-correct- 
ing code where e = (d — 1)/2 if d is odd, and is said to 
be an. e-error-detecting code where e = d/2 if d is even. 
We shall call either an e-error-correcting or detecting 
code a d-code. 

A d-code in S,(n) is said to be full if there is no d-code 
in S,(n) with more elements. For given n, d and k, let 
a,(n, d) denote the number of elements contained in a 
full d-code. A d-code is said to be close-packed if d is odd 
and if the union of closed balls of radius (d — 1)/2 about 
all members of the code exhausts S;,(7). 

Next let an operation + (mod k) be defined on S,(n) 
such that if s and ¢ are two members of S,(n), each letter 
of s + tis the sum modulo k of the corresponding letters 
of s and ¢. S,(m) 1s an abelian group under + (mod hk). 
Any d-code on S,(n) is said to be a group code if it is a 
subgroup of S,(m). A group code is full if there is no 
group code having more elements. The number of elements 
in a full group code is denoted by b,(n, d). 

Since the order of a finite group is a multiple of the 
order of every one of its subgroups and since the order of 
S,(n) 1s k", it follows that 

Lemma 1: If k is a prime, 6,(n, d) is a power of k. 


1 For a proof of this, see KE. Landan and A. Ostrowski, “On 
the diophantine equation ay? + by + ¢ = da,” Proc. London 
Math. Soc., ser. 2, vol. 19, pp. 276-280, 1921. 


8) IRE TRANSACTIONS ON INFORMATION THEORY 


It remains for us to specify the metric on S,(n). p is 
said to be the (czrcular) metric on S,(n) if for any pair of 


words = (2,.%,,> “" ; Sa), = Gi Ue, ee Olona 


n 


a p(xiy;) 


i=l] 


plz, y) = 


where 


poy) = Min (2: — Ge yee (mod k). 


The idea of the proofs for Theorems 1 and 2 runs as 
follows. We first show that a code can be constructed if 
a certain matrix can be found and then show how such 
a matrix can be obtained. 

Definition 1: An m by r matrix H(m, r) with entries a,; 
is said to be admissible if when its columns are regarded 
as words of S,(m): 


1) Every column is of distance more than one from 
the origin (0, 0, --- , 0). 

2) The sum or difference modulo k of any two columns 
shall be a word different from the origin. 


Tor example, let #,(2, 3) and (2, 3) be matrices 
O14 
S28 


0 1.2 
423 


2, 3) = E(2, 3)= 


? 


For k = 5, £,(2, 3) is not admissible since condition 1) 
is violated and /,(2, 3) is not admissible since condition 
2) is violated. 

Lemma 2: Let n be given and let m and r be any pair 
of positive integers such that m + r = n. Then for all 
odd k, an n-letter single error-correcting group code S 
having k” members can be constructed if an admissible 
matrix H(m, 7) can be found. 

Proof: Suppose H(m, r) is found, the theorem will then 
be proved by construction. Denote by S the set of all 


n-letter words of the form. 
ae RU yatta t- Ye) 


where 242 °+: 2, are arbitrary but 


y; = >, a,;4,(mod k), elt 2s eM. 
j=1 
with a;; « E(m, r). S has k” members. 
Let s, = (2:22 *** LrYWYo °°° Ym) ANd S. = (UU, --> 


U,V *** Um) be two distinct words of S. p(s:, s) 2 3 
if three or more of the x,’s are different from the corres- 
ponding w,’s. Otherwise, there are two cases. 

Case 1: x; = u; except for j = s, t, s ¥ t for which 
XL, FA Us, Ue FH Ut. 

Let us suppose that the letter distances p(x,, u.) and 
p(a,, U,) are each exactly 1, for otherwise p(s:, 82) would 
be larger than 2. Therefore, in this case, 


“%,—uU,=1 or K—1 (modk) 


June 


and 


4,— uw = 1 cor k— i Goode, 


Since H(m, r) is admissible, by condition 2 there exist 
p,q where p ¥ q, 1 S p,q S msuch that not both a,, + a, 
and a,, + a,, are zero (mod k) and not both a,, — a), 
and a,, — @,. are zero (mod k). We assert that either y, 
is different from v, or y, is different from v,. Suppose they 
are both identical, then 


—u,) + a,.(x, — uw.) = 0 
TES senile y Mee 9 = 0) 


contradicting condition 2 of admissibility of H(m, 1). 
Therefore p(s,, Ss.) 2 3. 

Case 2: x; = u; except for) = s for which 7, ~~, 

By condition 1 of admissibility of H(m, r), either there 
are two members a,,, @,, “ 0 or there is one member 
Pe Ua ant Fd |e Ab 

In the first instance, 


Aral (mod k) 


Ags(L, — 


Uy = Us = Oj Bs a Us) (mod k). 


Ul Ue Gets waar Us) 


First, suppose z, — u, = 2 ~ k — 1 (mod k). Then, 
since k is odd, it is impossible for either y, — v, = 0 or 
Ye — Y = O (mod k). Next, suppose x, — u, = 1 or 
k — 1 (mod k). Again it is impossible for either y, — v, = 0 
Or Y¥g — Ye = O (mod k). Therefore, p(s, s2) = 3. 

In the second instance, 


(mod k). 


Again since k is odd, if x, — u, = 2 ¥ k — 1 (mod &), 
y: — % ~ 0 (mod k). If s, — u, = 1 or k — 1 (mod b), 
then 1 < y, — v, < k — 1 implymg py, uv.) 2 280 that 
p(S1, S2) 2 3. 

Thus, in either case, two distinct words of S are at a 
distance at least three apart. Since it can be checked that 
S is an abelian group, the proof follows. 

In the case k is even, a slight modification in admissi- 
bility conditions is required. 

Definition 2: An m by r matrix H(m, r) is said to be 
E-admissible if E(m, r) is admissible and 

3) In every column j of H(m, r), there is an element 
a,;; such that 2a;; + 0 (mod k). 

Lemma 3: Let m, r be defined such that m + r = n 
where n is given. Let k be even. Then an n-letter single 
error-correcting group code S having k” members exists 
if an H-admissible matrix H(m, r) can be found. 

Proof: The only part of the proof which is different 
from the proof for Lemma 2 is in Case 2 where x; = u; 
except for 7 = s for which x, ¥ u,. Consider again two 
subeases. 

Case 2.1: There are at least two members a,,, a,, ~ 0 
in £(m, r) such that either 2a,, ~ 0 (mod k) or 2a,, ¥ 0 
(mod k) by H-admissibility of E(m, r). Suppose 20,, ¥ 0 
(mod k). Now 


Ye — Vn = Aiel&s = Us) 


Gps ig ts) (mod k). 
Ye Ute Oh, =e) 


rirst suppose x, — u, = 2 (mod k). Then since 2 a,, ~ 0 
mod k), y, — v, ~ 0 (mod k). Next suppose x, — u, = 1 
ir & — 1 (mod k). Then y, — v, ~ 0 and y, — v, ¥~ 0 
imod k). Therefore p(s,, s.) 2 3. 

Case 2.2: There is exactly one nonzero member a@,, in 
Ihe set {a,, | 7 = 1, 2, --- , m}. By H-admissibility of 
Hi(m, r), it follows that 1 < a,, < k — 1 and 2a,, ¥ 0 
mmod k). Now 

(mod k). 


Uh Cn ae iz Us) 


ifx, — u, = 2 (mod k), since 2a,, ~ 0 (mod k), it follows 
ee YY. — v: ¥ O (mod k). If x, — u, = 1 or k — 1 (mod 
fF), then 1 < y, — v, < k — 1. In either case p(s,, s.) 2 3. 
(his concludes the proof. 

Lemma 4: A code S, whenever constructable by appli- 
tation of Lemma 2, is close-packed (hence full) if 2m + 1 
is a power of k. 

Proof: Each word s in S is contained in a ball consisting 
pf 2n + 1 words of radius 1 about s and different words 
of S are contained in disjoint balls. Thus, there are 
&"/(2n + 1) = k’ balls the union of which exhausts the 
set S. 

| Lemma. 5: Let m, r be defined such that 


| k m 


and 


>2n+1, + <Qn+1 


fe SS he = lie 


‘An admissible matrix H(m, r) having rm members exist 
‘for all odd k. 

Proof: The lemma will be proved by construction. Let 
jus consider, for each j, j = 1, 2, --: , r, the m-tuple 
\(@y;, @2;, *** 5 Gmj;) aS a member of S,(m). Let S be a 
subset of S;,(m) consisting of all members of S,(m) except 
the closed ball of radius 1 about (0, 0, ---, 0). Define an 
equivalence relation ~ on S such that if s,, s2, &S, 8) ~ 82 
if s; = 8. or s, = s 5. Then any set of representatives 
(one from each equivalence class) has (k” — 2m — 1)/2 
words. Since 


k” — 2m — 1 
2 


Ze i = Ti = Pe 


choose r of these words arbitrarily to form a set of letters 
with rm members. Since this set of letters satisfies the 
two conditions of Def. 1, it is an admissible matrix and 
the proof follows. 

Proof of Theorem 1: Define m such that 
| 


en Dy tt ene a << On ed 


Set r = n — m. Then by Lemmas 2 and 5, an n-letter 
re eode S having k” words exists. 
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Suppose first 2n + 1 = k”. Then by Lemma 4, S is 
close-packed so that b,(n, 3) = k’. Since 


k” 356 


jig 
( + e < 
balm ro) a 3 


= 5) cha 


— Tet 
= ee 


it follows by Lemma 1 that 
b(n + 1, 3) = b(n, 3). 
Next suppose k” > 2n + 1 and k”* < 2n + 1. Should 
n be increased by 1, we still have 


kK” =2n+3 and k”* < 2n + 38, 


so that m remains unchanged. Thus r is increased by 1, 
implying an (n + 1)-letter code having k’"’ words exists. 
Now 


oy eed 


I 43 On 8 


bm + 1,3) < ee 
Therefore, it follows by Lemma 1 that 6,(n + 1, 3) = 
Rea cor 


b(n + 1, 3) = kb,(n, 3). 


Lemma 6: Let k be even. For a given m, an H-admissible 
matrix having at least 


1s (i) 2 2H) 


members can be constructed. 

Proof: Let us again consider, for each j, 7 = 1, 2, --+ , 7, 
the m-tuple (@:;, @2;, *** , Gn;) aS a member of S,(m). 
Let S be the subset of S,(m) consisting of all members 
of S,(m) except 1) the closed ball of radius 1 about 
(0, 0, --- , 0) and 2) all members s of S,(m) such that 

=) 
Sa 8 

Again define an equivalence relation ~ on S such 
that if s;,<s3, 65, Sy —~4s) if $; =“s, Ors; = s1,,, Phensany, 
set of representatives (one from each equivalence class) 
has 3 (k" — 2" — 2m) members since there are exactly 
2” + 2m members of S,(m) which are not in S. The set of 
letters which make up members of S is then the H-ad- 
missible matrix in question. 

Proof of Theorem 2, 1): Theorem 2, 1) follows from 
Lemmas 2 and 5. 

Lemma 7: Let k be even. For some given n, let b,(n, 2) be 
the size of the largest code in which the distance between 
every pair of points is not less than 2 (7.e., a single error- 
detecting code). Then b,(n, 2) = k”/2. 

Definition 3. A matrix E(m, r) is said to be E’-ad- 
missible if it satisfies condition 2 of Definition 1 and 
condition 3 of Definition 2. 

Lemma 8: For a given m, there are at least m(4(k" — 2”)) 
members in an £’-admissible matrix. 

Proof: The proof here is identical with the proof for 
Lemma 6 if we let the members of class 1 excluded from 
the set S in that proof be now included in S. 

Proof of Theorem 2, 2) and Corollary 1: Let m be given 
and let r = 3(k™ — 2”). Let S, be an r-letter single 
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error-detecting code with k’/2 members as given by 
Lemma 7. By Lemma 8, for the pair m, r in question, an 
K’-admissible matrix H(m, r) can be constructed. For each 
x = (21, %, °** , %) in S,, define y; such that 


Ui = SS A; ;U; 


Pol 


(mod k), 1 =) Pn: 
where each a;; belongs to the £’-admissible matrix 
E(m, r). Let S,,., be the set of all (m + r)-letter words 


(aes Ve, RS : 5) pe) 


» U,, Yiy Yo, * 


where (21, %2, °°: , X,) eS, and the y,’s are given above. 
Then S,,,, 1s a code satisfying the conditions of Theorem 
Do), 


INFORMATION THEORY Jun 
Next let S,,,2, be the set of all (m + 2r)-tuples 
(2, soa, sy Ley Yry Yor °° * » Ymy V1, Ve, * °° 5) 


where (21, %, °+: , v,) eS, and the y,’s are determined b: 
the linear relations above. S,,,2, has k'/2 members anc 
as in the proof of Theorem 1, we may show tha 
p(Si, S&) 2 5 for every pait si, Ss: 25,427) Also, iftsiae 
eSeixerj them Sy <b. sy 22S vope eieherefore’ mS 37am 
(m + 2r)-letter double error-correcting group code witl 
k’/2 members, and the proof of Corollary 1 follows. 
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CORRECTION 


Paul E. Green, Jr., author of ‘The Output Signal-to- 
Noise Ratio of Correlation Detectors,” which appeared 
on pages 10-18 of the March, 1957 issue of these Trans- 
ACTIONS, has requested the editors to make the following 
corrections to his paper. 

In the first paragraph on page 12, the term wu, in the 
last line should be p;. 

In (9) on the same page, Re([X(w) H(w)|*[X(w) H(w)]) 
should be [X(w) | H(w) |* X(w)] + Re([X(w) H(w)] * 
[X(w) H(@))). 

The right side of (11) on page 13 should be preceded 
by | I(A))’. 

In (12), on the same page, multiply the first denomi- 
nator integrand term X*(w) Re[H(w)| by a factor of two, 
and in (13) N2(w) should be replaced by N2(w) + A). 

The fifth from the last line in column two of page 13 
should read: 


For the bandpass filter, the reciprocal --- . Insert afte 
the last line: For the low-pass filter, the effective integra 
tion time is 7/W,. 

The following should be added to footnote 8: By usin; 
integration limits of — © and +o instead of 0 and o 
one obtains a different definition of W,, sometimes used 
which is the reciprocal of the effective integration tim 
for both types of filter. 

In (25) on page 15, h; in the second term should b 
replaced by h;, and should be &. 

On page 16, the first line after (85), the equation refer 
ence should be (28), not (27). 

On page 17, in the equation following (41), replac 
1/8 with 1/2. In the second paragraph of column one 
the equation reference on the fourth line should be (25 
instead of (3), and in (51), on the same page, the righ 
side should be multiplied by | Z(A) |’. 


Za 


Summary—tThe effect of noise upon a method of information 
mansmission and recovery is analyzed. The information is coded 
p the frequency shift of a carrier. The carrier is assumed to be 
1 te over two channels which have 90° phase difference. 

e recovery of the information is accomplished in a new way by 
punter” the frequency shift by means of a ‘‘coherent cycle 


unter.” 

A general expression is obtained for the expected frequency 
easurement of the coherent cycle counter in terms of the signal 
nd noise autocorrelation functions. The percentage bias of the 
unter is shown to be a function only of the signal-to-noise ratios 
| the two channels. In the noise-free case, the expected frequency 
ee is given by the centroid of the signal power spectrum. 
Yhen no signal is present the average indicated frequency is 
bro, thereby effectively cancelling the average effect of the sources 
noise in the system. For applications in which the transmitted 
sequency fluctuates about the carrier or reference frequency, 
ke method is shown to be superior to both an axis crossing counter 
nd an ideal phase differentiator when it is required to operate 
hrough zero frequency difference. 


INTRODUCTION 


PROBLEM of frequency measurement arises 

whenever information is transmitted over 
a nolsy channel as the frequency shift of a 
arrier.. ° Because of the random nature of the super- 
mposed noise which arises in the system components as 
‘ell as in the channel, the measured frequency is subject 
» random fluctuations. Additional fluctuations are 
resent when the modulating signal itself is random. 
fonsequently the frequency fluctuations must usually 
c smoothed to yield an average indicated frequency. 
When the modulating signal is sinusoidal, the desired 
utput is, of course, the sine wave frequency. When the 
rodulating signal is random, the desired output is some 
neasure of the central tendency of the signal power 
pectrum. 
One of the effects of noise being present with the signal 
1 the input to a frequency measuring device is to introduce 
bias into the average output. The bias for a particular 
evice is defined as the difference between the mean 


* Manuscript received by the PGIT, July 16, 1957; revised 
anuscript received, April 21, 1958. 
+ 3766 Acacia St., San Diego, Calif.; formerly with Ryan Aero- 
utical Co., San Diego, Calif. 
1p, M. Schultheiss, C. A. Wogrin, and F. Zweig, “Short- 
me frequency measurement of narrow-band random signals in 
ae presence of wideband noise,” J. Appl. Phys., vol. 25, pp. 1025— 
036; August, 1954. 

2H. Steinberg, P. M. Schultheiss, C. A. Wogrin, and F. Zweig, 
Short-time frequency measurement of narrow-band random 
gnals by means of a zero-counting process,” J. Appl. Phys., vol. 
6, pp. 195-201; February, 1955. 

3G, R. Arthur, “Statistical properties of output of a frequency 
sitive device,” J. Appl. Phys., vol. 25, pp. 1185-1195; September, 
954. 
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output when noise is present and the mean output when 
no noise is present. In general, the bias will be a function 
of the signal-to-noise power, the average signal frequency, 
and the noise spectral shape. Thus, for example, the 
average output of an axis-crossing counter is the radius 
of gyration f, of the signal power spectrum in the noise- 
free case. When noise is superimposed on the input signal, 
the average output is given by 


= P+ K’ 1/2 
eT i Mm) 


where K is the ratio of the radius of gyration of the noise 
power spectrum to the r. of g. of the signal spectrum, and 
P is the average signal-to-noise power. Consequently 
the percentage bias of the axis-crossing counter is 


Gye a Bae Ie a 
100) j. == (5 i ) i| x 100. (2) 


In a similar manner, the frequency bias of an ideal 
phase differentiator (a meter that measures frequency 
as the time rate of change of phase) when the input 
signal is a sine wave plus band-limited flat noise is* 


100 a = 100 (K — le” (3) 
where P and K have the same meaning as in (2) and f, 
is the sine wave frequency. 

From (2) and (3), it is seen that the bias of the two 
meters is zero when the parameter K is unity, that is, 
when the radius of gyration of the noise spectrum coin- 
cides with the signal radius of gyration. In other cases, 
the percentage bias is a function of both the signal-to- 
noise power and the value of the central noise frequency 
relative to the signal frequency. 


CoHERENT CycLE CouNTING 


In some applications it is required to measure the 
frequency shift of a transmitted signal whose instantaneous 
frequency f, fluctuates about a carrier or reference 
frequency fr. If the transmitted frequency is greater 
than the reference frequency, the frequency difference is 
regarded as positive; if less the difference is negative. The 
problem in this case is therefore not only to measure the 
difference between the transmitted and reference fre- 
quencies, but also to detect the algebraic sign. 


48. O. Rice, ‘Statistical properties of a sine wave plus random 
noise,’’ Bell Sys. Tech. J., vol. 27, p. 129; January, 1948. 
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The following assumptions will be made with regard 
to the transmitted signal: 


1) The transmitted frequency fr lies in a frequency 
band about the reference frequency fp. 

2) It is possible to receive the transmitted signal in 
two channels which have 90° phase difference and 
in such a way that the noise sources in the two 
channels are uncorrelated. 


Under these two assumptions, the transmitted infor- 
mation can be recovered by a new method of frequency 
measurement, described as ‘coherent cycle counting” 
in which the expected percentage bias depends only upon 
the signal-to-noise ratios in the two channels. In addition, 
the coherent cycle counter will detect the sign of the 
frequency deviation from the reference signal frequency. 

A block diagram of the information transmission and 
recovery is given in Fig. 1. The coherent cycle counter 
has as its inputs the signal plus noise waveforms X,(t) 
and X,(¢). 


Noise 
Receiver LU Mixer 
e 


ransmi 


Fig. 1. 


When the transmitted signal frequency exceeds the 
reference frequency, the phase of the signal in the second 
channel will be advanced 90° with respect to the signal 
phase in the first channel. When the transmitted frequency 
is less than the reference frequency, the phase difference 
of the signal in the two channels will be — 90°. To show 
this, suppose the transmitted signal is given by 


S7(é) — Ar sin (wrt Se wWpt + 6,) (4) 
and the reference signal is given by 
Sp(d) = Arp sin (wrt + 65). (5) 


Then the mixed and filtered signal in the first channel 
will be proportional to cos (+ wpt + 6, — 62) or cos 
(wpt + 6, * 6). On the other hand, the limiter input 
in the second channel will be proportional to cos (+ wp 
t+ 6, — 6. — 2/2) or cos (wp t + 6, F 6. F 2/2). Conse- 
quently the phase difference of the signal between the 
first and second channels is plus or minus 90° according 
as the transmitted frequency is greater or less than the 
reference frequency. 

The limiter inputs in the two channels can therefore 
be represented by 


X,(t) = S,(2) = N,(t) (6) 


Jun 


X,(t) = S(t) + N2(t) (7 


where the frequency of the signal jS,(é) is equal to th 
difference between the transmitted frequency and th 
reference frequency, S,(t) is equal to S,(¢) with a 90 
phase-shift, and N,(t) and N,(t) are the noise voltage: 
present at the input to the limiters in the first and secon¢ 
channels, respectively. 

The output of the pulse shaper is a train of positive 
and negative narrow pulses of equal area, the position: 
of which along the time axis correspond to zeros of the 
signal plus noise X,(t). Whether a pulse has positive o} 
negative polarity depends upon the slope through zer« 
of X,(t). The sign of a pulse is then reversed in the outpu 
of the multiplier if the limiter output in the seconc 
channel is negative. When the limiter output is positive 
the pulse passes through the multiplier with no change 
polarity. The desired weighting function of the low-pas: 
filter following the multiplier is given by 


Ryan / 2) Oe eT 
= 0, 


I 


(8 
otherwise. 


Consequently, the desired output at time ¢ of the low: 
pass filter following the multiplier is given by the following 
functional: 


F(X, X) = oe i _ AX) XI BuX,) — Ide @ 


where the prime notation denotes differentiation witk 
respect to time, 6(X) is the Dirac delta function, and u(X- 
is the unit-step function. The interpretation of (9) is as 
follows: let N“(t, 7) denote the number of times the 
compound event 


{X,(t) = 0 and Xi(t)-X,.(t) > 0} 


occurs in the interval ({ — 7’, ¢); similarly let N (t, 7 
denote the number of occurrences of 


{X,() =0 and X;(d)-X.(t) < 0} (11 


(10 


in (¢ — 7, ¢). Then 


N+, T) = / i o(X,)X(X%, X,) dt a 
and 
N-U,1) = | 7 6(X,)Xiw(X!, X,) dt. (13 
for almost all realizations X(t), X.(t), where 
2X1) a) sO) eee eae (14 
w(X{, X2) = ul Xt) — u(X,). (15 


A rigorous proof of (12) and (13) can be made throug] 
the use of the Schwartz theory of derivatives.” For ou 


°L. Schwartz, “Theorie des Distributions,’ Hermann, Paris 
France, 1950. 


poses it suffices to note that the integrands of (12) 
(13) are zero everywhere for almost all realizations 
X,(t) and X,(t), except for unit impulses coinciding 
‘time with positive and negative pulses, respectively, 
‘he multiplier output. 
from (12)—(15), the functional (9) can be expressed as 
XG Xo) = sf LN "(head craven tn) | (16) 
(3 the weighting function of the low-pass filter is 
n by (8), the frequency indication at time ¢ is 


Volts di) =r Netty) — Nd, 2). (17) 


= 


or stationary processes, the average indicated 
Kuuency is independent of the time of observation ¢ 
t depends on the time constant 7 of the counter. Taking 
‘ensemble average of (17) one finds for ergodic processes 
ih ae z 
(feelT)) = ox KN (T)) — (N())1. (18) 
The required ensemble averages (N‘(7’)) and (N (T)) 
h be found by calculating the expectations of the 
actionals (12) and (13), respectively. For simplicity 
motation let x, = X,(t), ~% = X{(t), and x; = X,(é). 


jen. 
H(D)) = Ih i AGN EES me eta) 
ray = ff aedzsuter, es) de der, a, %) 20 


| 

lére oy, 2) = jomt probability (ai < wa. < 
xz, < z} and R is the three-dimensional probability 
ace of the random variables x,, v2, and 23. 

Before (19) and (20) can be evaluated, one needs to 
w the statistics of the extracted signal plus noise 
veforms X,(t) and X,(t). The analysis is conveniently 
‘ided into two eases: first, when the extracted signal is 
1dom, and second, when the extracted signal is a sine 


se 1—Random Transmatied Signal 


he first case assumes that the extracted signals S,(¢) 
d S(t) appearing at the limiter inputs in the first and 
ond channels, respectively, are stationary normal 
adom variables with power spectral density W,(f). The 
ise voltages V,(t) and N,(t) at the limiter inputs are 
o assumed to be normally distributed with spectral 
nsities W,(f) and W 2(f), respectively, and are un- 
related to each other and to the extracted signals. 
Assuming that the first order moments of S;(¢) and 
(t) i = 1, 2, are all zero, the joint probability density 
the maidor variables x1, 22, %3 is given by® 


6H. Cramer, pe Reece Methods of Statistics,’ Princeton 
iversity Press, Princeton, N. J., p. 311; 1946. 
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BAGG tions eee 


3 
“exp ‘5 | - | Pe, M 052 i dx, Ctx ax; (21) 
where the M,; are the cofactors of the moment matrix 

= || wi; || | MW | = determinant of M, and u,; = co- 
variance of x; and 2;,. 

The elements p,;; of the moment matrix M can be 
determined by the use of Rice’s expansion method’ or 
by the use of the spectral representation formula.* When 
these calculations are performed, the moment matrix 
turns out to be 


Wo + a 0 0 
M = 0 — YW! — as’ F2rypof, (22) 
0 Fro. Yo + Bo 


where y, is the signal autocorrelation function, a, and 
8, are the noise autocorrelation functions in the first and 
second channels, respectively, and f, is the centroid of 
the signal power spectrum; that is, 


¥, = EYSOSi(t + 7)} = B{SOS2¢ + 7)} 


z if “WG aoe ees 


a, = BINONE + 9} = [ Wil) cos 2xf af (24) 


Bae BLN AON te ee if WW icok pf ae 


j= [ SW) aff (26) 


where the right-hand integrals result from the familiar 
Wiener-Khintchine theorem.? The plus or minus sign 
prefixing f, in (22) depends upon whether the transmitted 
frequency is greater or less than the reference frequency, 
respectively. From (22) it is apparent that M,, = M,; = 0. 

When (21) is inserted in (19) and (20), the integration 
with respect to x, yields 


(N“)) = @n)** | MM 


| / | PEON CD nota herve 2X3) adie dx; dt (27) 
t-—T — co —o 
and 
(N(T)) = (29) °? | M |” 
t o>) os) 
. ‘? ik i! CEoU ne la) Ed (lon ee en Cate (28) 
t—T J—w J—o@ 
78. O. Rice, “Mathematical analysis of random noise,” Bell 


SUSmullechmde, vol. 23, pp. 306-310; July, 1944. 

Bn iby: Doob, “Stochastic Processes,” John Wiley and Sons, Inc., 
New York, N. aye » Dp. 027; 1953. 

oN. Wiener, “Generalized harmonic analysis,’ Acta. Math., 
vol. 55, pp. 7 258; 1930. 
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where we have set 


( : 4 ) : 
H(a,, 25) =exp {9 ' “4 = (Mats + Mss + 2M ssttats) f (29) 
Meecha 


Recalling the definitions of v(a,, w3) and w(a, #3) 
from (14) and (15), and noting that H(— 2, — 23) = 
H (a5, x3) and H(—,, 43), = A (a, a3) 2rsandas) 


can be simplified as follows: 


(N*(T)) = 22a ren | M mae 


: il | if Ri) dade ed ao) 
t—T 0 0 


(N-(T)) = 202m)? | M |?” 


: /[ | i CoH (3,05) OF te ae. (31) 
/t-T JO 0 


Eqs. (80) and (81) are similar to multiple integrals 
evaluated in the statistical study of certain nonlinear 
devices.'’'’' Thereupon the expected number of positive 
impulses counted in an interval of length T is 


|\M\T 


NL 32) 
ee Qe VM VW MoMs3 + Mos) 
and the expected number of negative impulses is 
ies MAST 
ON NCLB) pos EM (33) 


On VW Mol VMasMox — Mos) 


Consequently the average indicated frequency of the 
coherent cycle counter, after insertion of (82) and (33) 
into (18), is seen to be 


—|M|M,, 
2a V M2(Mo2.Ms ae M¢s) 


(fee) = (34) 


When the appropriate cofactors in (19) are evaluated 
and inserted into (34), the average frequency indication 
of the coherent cycle counter is 


| PP; 


where P, = Yo/ao and P, = Y/B> denote the signal-to- 
noise ratios in the two channels, respectively. 

In the absence of noise in both channels, (35) shows 
that the average indicated frequency of the coherent 
cycle counter is given by the centroid of the signal power 
spectrum. 

When no signal is transmitted, the cofactor M,, in 
(22) vanishes, making (N“(T)) = (N (T)) in (82) and 
(33). Thus in the no-signal case, the coherent cycle counter 


(35) 


0S. O. Rice, “Mathematical analysis of random noise (con- 
cluded),”’ Bell Sys. Tech. J., vol. 24, pp. 67-69; January, 1945. 

uJ. H. Laning, Jr., and R. H. Battin, “Random Processes in 
Automatic Control,’? McGraw-Hill Book Co., Inc., ‘New York, 
N. Y., Appendix A; 1956. 
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has the desirable property that its average output» 
zero. This result is independent of the spectral shape « 
the noise waveforms in the two channels. 

In the absence of noise in both channels, one can verif 
from (32) and (33) that the sum of the expected numbe 
of positive and negative pulses in the output of th 
multiplier reduces to Rice’s result for the expected numbe 
of zeros of a random process: 


(N*(T)) + (N (T)) = -. ae , noise-free case. (3¢ 


Iq. (35) shows that the percentage frequency bia 
depends only upon the signal-to-noise ratios in the tw 
channels. Furthermore, since the phase difference in th 
two channels is +90° or —90° according as the trans 
mitted frequency is greater or less than the reference 
frequency, the coherent cycle counter also acts as a sig 
detector, giving an average positive or negative voltag 
according to sign. 


Case II—Sine Wave Signal 
When the incoming signal is a sine wave 
S\(t) = Asinoat (37 


the joint density of the random variables x,, 2, and a 
defined above is given by’ 


APG, ta) = Or) 


ih ee eee ‘ coal 
“exp \; | M | pe M ,,(x; = E(x; ae ay, 
“dx, dx» da; (38 
where #, = A sin wt, #, = Aw cos wt, and 7, = + 
COS wt. 
The moment matrix is 
Ao 0 0 
Wl = MO =e (3¢ 
0 0 Bo 


where a, and 8, are given by (24) and (25), respectively 
The average value of (9) gives the expected indicate 
frequency: 


(f..) = 20) / il it 8(2,)es(Qu(ats) — 1)(2n)-*”? | ME |? 
et a NW | 2 
“exp ea [Mii(a, — %)° + M,,(x2 — 4.) 

+ M;,(a, — ny} dandy. dt0.be (AC 


® Rice, “Mathematical analysis of random noise (concluded) 
op. cit., p. 54. 


MEASURED FREQUENCY 


SINE WAVE FREQUENCY 


mi AVERAGE 


AVERAGE SIGNAL POWER 
AVERAGE NOISE POWER 


z. 2—Frequency measurement of coherent cycle counter for 
sine wave signal plus Gaussian noise. 


‘The integration in the 2’s yields 


pte See 
te? — Vee 
Tr 


ih X2 exp 2/M|™ Ip | M | as | at (41) 


nere we have introduced the standard normal distribution 


| $(2) = (2r)'\” i ele dt. 


‘When the appropriate cofactors of (39) are inserted 
ito (41), we have 


= saa ‘he t 
iv DAR TT 0 


-exp (—P, sin’ wt)¢(+ V2P, cos wt) dt (42) 


nere P, = A’/2a, is the average signal-to-noise power 
io in the first channel and P, = Ae} 285 is the average 
Re in the second channel. 

As the integrand of (42) is periodic with period 1/2f, 
nere f = w/2r, the integration can be simplified: 


7 = ip i; dv cos v exp (—P, sin’ v)6(+ V2P, cos 1) 


Me 2 124)" i. due?" o(0/2P,(1 — 2°)). (48) 


iq. (43). shows as in the case of a random signal that 
le average indicated frequency of the coherent cycle- 
punter is independent of the shape of the noise power 
vectra and is a function only of the signal frequency and 
1e signal-to-noise ratios P; and P, in the two channels. 
1 the no-signal case (P; = P, = 0), the average indicated 
equency is again zero. 
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Fig. 3—Frequency measurement of coherent cycle counter for 
Gaussian signal plus Gaussian noise. 


AVERAGE SIGNAL poweR 
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Fig. 4—Average frequency measurement of zero counter for Gaussian 
signal plus Gaussian noise. K = average noise frequency /average 
signal frequency. 


Unfortunately when P > 0, (43) cannot be integrated 
in closed form, so that numerical methods have to be 
resorted to. Fig. 2 shows the results of the numerical 
integration for the case P, = P,, where the ratio of 
average indicated frequency to signal frequency is plotted 
vs average signal-to-noise power. 


CONCLUSIONS 


When a random signal is transmitted, the coherent 
cycle counter can be compared with the axis crossing 
counter by means of (1) and (35). Eq. (385) shows that the 
bias of the coherent cycle counter can be made zero by 
inserting a gain equal to 


A + DP, + 1) 
IP IES 


The equivalent operation of the axis crossing counter 
requires knowledge of the noise spectrum as well as of the 
average signal-to-noise ratio. The expected frequency 
measurement of a random signal by the coherent cycle 
counter and the zero counter are given by Figs. 3 and 4, 
respectively. 
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Similarly, when a sine wave signal is transmitted, the 
gain required to eliminate the bias is dependent only on 
the average signal-to-noise ratios, as is shown by (48). 
lor an idealized phase differentiator (2) shows that the 
gain adjustment is dependent both on the noise spectrum 
and the average signal-to-noise power. Figs. 2 and 5 give 
the expected measured frequency of a sine wave perturbed 
by Gaussian noise for the coherent cycle counter and the 
phase differentiator. 

Finally, the coherent cycle counter also acts as a sign 
detector, that is, it specifies whether the average trans- 
mitted frequency is greater or less than the reference 
frequency. This fact allows the coherent cycle counter to 
operate through zero frequency difference between the 
transmitted and reference signals. 
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A Comment on the Optimum Non- 
linear Filtering of Balakrishnan 
and Drenick 


In a recent paper, Balakrishan and 
Drenick derived optimum nonlinear filters 
applicable to strictly stationary discrete 
signals.! Their filtering procedure consists 
of three steps in cascade. First, a linear 
operation changes the input to a new 
process possessed of a white spectrum. A 
nonlinear zero memory operation is then 
performed. Finally, another linear filter 
restores the original spectrum to the output. 

The above mentioned linear operations 
relating the input process 2, to the white 
process {, can involve only present and 
past values of x, and ¢, if they are to be 
realizable. These operations are therefore 
given by Balakrishan and Drenick in the 
form 


Lt, = SD DiGnei (1) 
0 


1A. V. Balakrishan and R. Drenick, ‘‘On optimum 
nonlinear extraction and coding filters,’’ IRE Trans. 
on InrorMATION THEORY, vol. IT-2, pp. 166-172; 
September, 1956. 


and 


& a DS ae (2) 
0 


That the representation (1) and its inverse 
(2) actually exist is an assumption basic 
to the referenced paper. No attempt was 
made there to explore the conditions under 
which (1) and (2) are valid, and if so, how 
the coefficients a; and b; are to be evaluated. 

This note will indicate that (1) is valid 
under very general conditions, and that 
whenever (1) is true, (2) holds also. In 
addition, coefficients of (1) and (2) are 
determined. 

The existence of (1) is assured if we 
eliminate from consideration inputs whose 
future can be predicted precisely from their 
past.2 Such inputs are called deterministic, 
and are of little interest in filtering and 
prediction theory. Likewise, if an input is 
composed of a deterministic and non- 
deterministic part, we are not concerned 
with the deterministic (perfectly pre- 
dictable) portion. 

An input process is wholly nondeterm- 


2See J. L.-Doob, “Stochastic Processes,’ John 
Wiley and Sons, Inc., New York, N. Y., pp. 577-579; 
1953. 
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Fig. 5—Average frequency of sine wave plus band-limited “whit 
Gaussian noise. K = average noise frequency/sine wave frequen 


istic in the sense just described [and ( 
is valid] if and only if the spectral densi 
©(f) exists (without delta functions) and 


+1/2 
i log &(f) df>—@. “{ 


1/2 


This requirement is analogous to t 
Paley-Wiener condition needed for filteri 
and prediction of continuous parame’ 
random processes. 

A unique determination of the constar 
a; is now possible in terms of log 
Again, the proof is omitted here.? We ha 


[ow 0p a 


=1y2 


1 
ao = exp 9 


if the ¢, have been normalized so that thi 
mean square value is unity. The remainc 
of the a; are defined through the relatic 
ship 


SoZ erexp E +2>0 oa! | ( 
0 1 


where z is complex with | z| < 1, and t 
cr, are computed from 


Ly 2 
C, = al e"*™ log &(f) df. ( 
SIP} 


o determine a given a; from (5), it is 
y necessary to differentiate the right- 
nd side of the equation 7 times in z, set 
. 0, and divide by 7!. Note that a; is a 
action only of those c; whose index k < j. 
erefore, finding the first m a; means 
M only the first m cz need be obtained. 

he question of the number of a; which 
puld be computed is answered as follows. 
cause ¢; 18 an orthonormal sequence and 
} converges in probability mean, 


Bion. | eae So anes? (7) 


that a finite memory filter utilizing only 
p first m coefficients incurs a mean 
liare error of 


m—1 


tle, |— > lal. 
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In practical applications, the residual 
error must be balanced against the ad- 
ditional complexity resulting from the use 
of a greater number of terms. 

Turning now to (2), we note that ¢, 
fr-1, (n-2 *** Constitutes a basis in the 
manifold of 2p, tn-1, In-2 *** . It follows 
that ¢, must be a linear combination of 
those x, for which n > k; this is exactly 
the form (2). 

The coefficients b; of (2) are most easily 
obtained in terms of the a;. In the first 
place, multiplying (1) by ¢,-, and taking 
the ensemble average yields 


DON ONC) aa iE (8) 


Performing the same operation on (2) and 
substituting (8) then gives 


k 
Ds b;Ox-; = Sox (9) 
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for k = 0, 1, 2, --: . This set of equations 
can be solved, for example, by starting 
with k = 0 and then proceeding to equations 
of higher & consecutively. Thus, in each 
equation 6, appears in terms of bx_1, 
bps, +++ which have already been found. 

To determine how many 6; are needed to 
approximate ¢, satisfactorily, we again 
compute the error when m such 6’s are 
used in a finite memory filter. This error is 


mi m-1 


t= >> >> 0b. RG See) 


7=0 k=0 


where /¢(p) is the correlation function 
E( 210740). 
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Bargellini, P. L. 
Beane, T. E. 


Bensky, L. 8. 
Berkowitz, R. S. 
Bicking, H. P. 
Booker, R. W. 
Booth, A. T., Jr. 
Bradford, C. E. 
Bradley, W. E. 
Bruchlos, H. O. 
Bucher, T. T. N. 
Bushnell, R. H. 
Bycer, B. B. 
Byers, A. C. 
Carter, W. S., Jr. 
Charton, Ss. 
Chronister, IW, M. 
Coffey, E WwW, 
Cohen, I. 
Cohen, P. 
Colodny, S. H. 
Cornell, J. A. 
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Courtney, J. E. 
Cramer, B. G. 
Crawford, CaASar 
Curtis, W.C. 

De La Cuesta, H. 
Deutsch, J. 
Deutsch, R. 
Ehrich, W. G. 
Elizondo, E. L, 
Fabbioli, L. F. 
Fenton, F, H,, Jr. 
Fischbeck, K. H 
Flomenhoft, M. 
Foley, G. M. 
Friend, A. W. 
Fuchs, A. M, 
Gantert, A. 
Garber, H. N. 
Gaynor, E. 
Geselowitz, D. B. 
Goblick, T. J., Jr. 
Gollub, R. 
Goodrick, R. H. 
Gottschalk, J. M. 
Gray, Hed, ute 
Greenfield, A 
Halpern, H, M. 
Harris, W. A. 
Hartnett, E. J. 
Hellerman, H. 
Herscher, M. B. 
Hitt, J. J. 

Hoger, D. T. 


Holshouser, J. R., Jr. 


Honda, H 
Howery, R. W. 
Hurford, W. L 
Ingerman, P. Z. 
Joseph, D 
Joshi, A. K. 
Kanal, L. N. 
Kashtan, § 
Katz, E. 8. 
Koos, E. E. 
Kozikowski, J. 
Hreadel 2. S 


Lockhart, Abtes 
Lynch, C. View ors 
Mah, L. 

Maron, I. 
Mathes, R. E. 
Mauchly, J. W. 
MceCollor, R. L. 


McCracken, L. G., Jr. 


McKinney, R. S. 
Mertens, L. E. 
Milewski, C. A 
Mittelman, W. 
Newman, E. L. 
Noel, R. Y. M. 
Osbahr, B. F. 
Patterson, G. W. 


Rogers, R. F. 
Rojas, R. R. 
Roop, R. W. 
Rosen, G. 
Rosenzweig, G. FE. 
Rother, F. 
Ryan, V. F., Jr. 
Safren, HG: 
Saloon, J. A. 
Scholtes, PW: 
Shennoy, Ree: 
Sherman, 8S. M. 
Showers, R. M. 


picinberes BaD: 
Sublette, I. H. 
Taenzer, E. 
Taylor, J. R. 
Tompkins, H. FE. 
Tweet, B. O., Jr. 
Urkowitz, H. 
Van Gelder, A. 
Walsh, D. E. 
Weiner, J. R. 
Weinger, R. 
Weisbecker, J. A. 
Weiss, BE. 
Wilcox, R. C. 
Williams, A. J., Jr. 
Woerner, L. G., Jr. 
Wolin, S 

Woll, H. J. 
Wong, S. Y. 
Yamada, H. 
Yang, “1. 
Zebrowitz, 8. 


Virginia 


Conable, J. H. 
Corpening, A. 
Dial, E. W. 
Fossum, T. T. 
Georgallis, G. C. 
Harris, O. R. 
Harvey, G. L. 
Hastings, C. E. 
Maxwell, M. S. 
Morton, R. W. 
Noble, L. V 
Premo, D. A. 
Taylor, M. L. 
Welch, A. A. 


Washington, D. C. 


Alderson, W. 8S. 
Alexander, H. R. 
Alexander, S. N. 
Allen, D. A. 
Anders, F’. W. 
Ballard, A. H. 
Bareau, A. R. 
Bauer, P. S. 
Baxter, J. L. 
Bernstein, B. 
Blackburn, C. A. 
Blair, C. R. 
Boggs, G. E. 
Brennan, L. E 
Bullock-Webster, M, 
Bush, G. B. 
Campanella, S. J. 
Cleckner, D. C. 


Finney, W. J. 
Fisher, R. H. 
Fleming, J. J. 
Flubr, F. R. 
Flynn, J. B. 
Freeman, J. J. 
Gale, M 
Gaze, R. H. 
George, S. F. 
Gleason, R. F. 
Godsey, W. J. 
Goldberg, H. 
Grisamore, N. T. 
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oe 


Katzin, M. 

King, A. M. 
King, W. P. 
Kirsch, R. a 
Kirshner, J. 
Klein, M. i 


Kullback, 8. 
Laine, R. O. 
La Pointe, J. C. 
Leiner, AL. 


Lieberman, G. 
Loda, C. J. 

Lun, M. J. 

Mallin, J. A. 
Mauldin, H. W., Jr 
McClurg, G. H. 
McGinnis, C. FE. 
McLaughlin, D. J. 
McLeod, J. 8. 
Melton, B. S. 
Metzger, W. 
Mitchell, G. J. 
Morscher, L. N., Jr. 
Morse, M. 8. 
Neumann, A. J. 
Norton, M. H. 
Notz, W. A. 

Ould, R. 8. 

Paden, D. R. 
Page, R. M. 
Peterson, H. L. 
Petrie, G. W., III 
Petritz, R. L. 
Peyton, P. B., Jr. 
Phillips, M. L. 
Pitsenberger, J. W. 
Poor, V. D. 
Reagen, E. J. 
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Reed, 8. F. 
Regardh, C. B. 
Reiser, D. 
Robel, R. B. 
Rochelle, R. W. 
Rotkin, I. 
Runyan, R. E. 
Safford, L. F. 
Schwartz, R. J. 
Scott, R. M. 
Scott, S. R. 
Sencenbaugh, DAWe 
Shapiro, G 
Sharp, N.S. 
Shepard, D. H. 
Singer, C. H. 
Smith, B. D., Jr. 
Smith, E. D. 
Spetner, L. M . 
Stitch, B. D 
Summers, C. R. 
Swiercezek, W. L. 
Swramik, _ Poa bh 


Van Lunen, R. D. 
Wagner, F., Jr. 
Wald, B. 
Waldschmitt, J. A. 
Walsh, J. P. 
Weihe, W. K. 
Wilcox, R. H. 
Willard, J. M 
Winkler, S. 
Worne, B. E. 
Yost, W. E., Jr. 
Youden, Ww.w. 
Zirm, R. R. 
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Akron 


Brown, L. C. 
Burke, J. T. 
Carpenter, C. P. 
Diamantides, N. D. 
Fordham, C. E. 
Horowitz, M. 
Huffman, D. L. 
Kelly, C. M. 
Kierstead, F. H., Jr. 
Kult, M. L. 

Miller, J. H. 
Pressel, P. I. 
Steigerwalt, O. I. 


Central Pennsylvania 


Baker, W. L. 
Bowhill, S. A. 
Harvey, H. B. 
Higdon, R. V. 
Key, C. L., Jr: 
Knausenberger, G. E. 
Lawther, J. M 
Lemley, L. W. 
Lopez, A. F. 
Miller, N. B. 
Norris, RS: 
Stein, 8 
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Berg, D. F. 
Doerr, W. H. 
Kaufman, B. A. 
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Gilbert, E. G. 
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Kilmer, W. L. 
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Lindahl, C. E. 
Lindsay, W. J. 
Lippmann, S. A. 
Macnee, A. B. 
McGlinn, E. J. 
McNabb, J. W. 
McPherson, R. R. 
Morgan, B. S., Jr. 
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Nakagawa, N. 
O'Neal, R. D. 
Otterman, J. 
Peterson, G. EB. 
Piper, C. A. 
Porcello, L. J. 
Rauch, L. L. 
Reiher, H. F. 
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Robinson, G. H 
Scott, N. R. 
Steinmann, W. L. 
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Tieman, C. R. 
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Webb, H. E. 
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Cedar Rapids 


Babcock, J. H. 
Babillus, J. 
Kemble, T. H. 
Leverington, R. D 
Lowenberg, E. C. 
Mansur, G. F., Jr. 
Stover, H. A. 
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Arsem, A. D, 
Beam, R. E. 


IRE TRANSACTIONS ON INFORMATION THEORY 
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Bobis, J. P. 
Borrowman, J. H. 
Boyd, D. M., Jr. 
Brauer, H. H. 
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Carlson, G. R. 
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Chen, C. L. 
Chorney, P. L. 
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Condron, W. F. 
Costa, P. J. 
Dawson, J. W. 
Demasy, J. 
Druz, W. S. 
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Franklin, C. W. 
Gerlach, A. A. 
Hansen, T. A. 
Henebry, W. M. 
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Hupert, J. J. 
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Jenness, R. R. 
Jones, A. H. 
Jones, R. W.. 


Ma, H. J. 
MacDonald, W. F. 
Magnuski, H. 
Mansfield, R. 
Marshall, B. O., Jr. 
Messinger, H. P. 
Mikulski, J. J. 
Mittelmann, FE. 
Mittra, R. 

Moe, M. L. 

Moon, R. J. 
Morrison, P. 
Mueller, F. M. 
Muerle, J. L. 
Mullin, A. A. 
Nordyke, H. W., Jr. 
Peach, L. C. 

Pye, H. C. 
Rabowsky, I. 

pac erds, i F. 


Rohr, W. 
Heeani, E. 
Rubinfien, D 
Ruina, J. 
Sawada, F. H. 
Sayles, H. L. 


Schulz, R. B. 
Sherrick, D. C. 
Skaperdas, D. O. 
Sladkey, R. J. 
Soma, R. 
Sommeria, M. R. 
Splitt, F. G., Jr. 
Stastny, G. F. 
Stewart, C. H., II 
Thomas, R. G. 


Van Valkenburg, M. E. 


Waller, E. E. 
Warnke, G. F. 
Wavering, A. J. 
Webb, H. D. 
Wei, L. Y. 
Weissman, R. M. 
White, E. 8 
Woveris, L. J. 
Yeslin, A. R. 
Zakhaim, M. 


Fort Wayne 


Choi, G. Y. H 
Clark, J. R. 
Hessler, J., Jr. 


Indianapolis 


Cooper, G. R 
Masnaghetti, TR, Ee. 
Schultz, F. V. 


Toure 
Louisville 
Kain, R. Y. 
Milwaukee 


Asmuth, J. L. 
Benedict, T. R. 
Chapman, D. M. 
Davidson, C. H. 
Gebhard, R. F. 
Glass, T. J. 
Keenan, J. A 
Kushner, H. J. 
Limpel, E. J. 
Mrazek, D. A. 
Norum, V. D. 
Rideout, V. C. 


Scheibe, E. H, 
Schlager, K. J. 
Schweppe, F. C. 
Stephan, R. R. 
Theiss, C. M. 
Tyson, H. B. 
West, B. M. 


Omaha-Lincoln 


Bashara, N. M. 
Chamberlain, I. J. 
Heim, K. E. 
Wycoff, K. H. 


South Bend-Mishawaka 


Bardell, P. H., Jr. 


Twin Cities 


Bratschi, R. W. 
Cohen, A. 
Featherstone, R. P. 
Frohbach, H. F. 
Grosz, W. 8S. 
Hardenbergh, G. A. 
Hawley, C. L., Jr. 
Holte, J. E. 
Ludwig, J. T. 
Nordstrom, J. E. 
Raabe, H. P. 
Robertson, AY dels 
Sanders, R. M. 
Schmitt, O. H. 
Schuck, O. H. 
Schultz, S. W. 
Thompson, R. H. 
Wiedman, R. FE. 
Wilson, A. P. 
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Dallas 


Brachman, M. K. 
Cawood, J. L. 
Erringer, O. W., Jr. 
Fain, W. W. 
Fuller, W. D. 
Kettler, C. L. 
Leming, T. L. 
Logan, J. J. 
Miller, J. C. 
Mitchell, W. R. 
Mut, S. C. 
Sanford, A. L. 
Sarrafian, G. P. 
Smith, C. H. 
Stanton, A. N. 
Strom, L. D. 
Wadel, L. B. 
Weedfall, W. W. 
Wilhelm, E. S. 
Ziemer, D. R. 


Denver 


Burkhard, D. G. 
Cohen, R. 8. 
Cook, E. E. 
Coombs, W. C. 
Cottony, H. V. 
Daniels, W. H. 
Rice, R. B. 
Slutz, R. J. 
Stacey, D. 8. 
Sugar, G. R. 
Wedge, T. E. 
Zanboorie, M. H. 


El Paso 


Carbine, I. L. 
Kidwell, R. P. 
Pyle, C. A. 


Fort Worth 


Brust, M. F. 
Delaney, J. S. 
Dodd, J. M. 
Fletcher, C, H. 
Hix, EB. W. 
Sissom, A. W. 
Tedeschi, A. 


Houston 


Bobbitt, J. T. 
Easterling, M. F. 
Jones, H. J. 
Keating, L. M. 
Kolb, R. H. 
Pierson, A. L., IIT 
Rust, W. M., Jr. 
Schneider, W. P. 
Smith, F. C., Jr. 
Suttle, A. D., Jr. 
Tanguy, D. R. 
Tyler, W. L. 
Wischmeyer, C. R. 


Kansas City 


Cummings, A. J. 
Findley, L. D. 
Halijak, C. A. 
Love, B. E. 

Van Der Maas, G. J 
Wilcox, J. V 


New Orleans 


Cronvich, J. A. 
Gordon, FE. 
Hanle, R. L., Jr. 
Hatfield, F. A. 
McLean, L. V. 


Oklahoma City 
Puckett, T. H. 


St. Louis 


Cummings, G. M. 
Furfine, A. L. 
Hirsch, O. C. 
Hobbs, E. W. 
Keiser, B. E. 
Kline, R. M. 
Little, G. R. 
Mohrman, R. F. 
Tucker, M. F. 
Watts, C. A. 
Zaborszky, J. 


San Antonio 


Anderson, W. L. 
Bostick, F. X 


Duesterhoeft, W. C., Jr. 


Economy, R 

Garner, W., Jr. 
Hoffman, A. A. J. 
King, J. D. 
McQuown, A. N., Jr. 
Smith, 8. E. 

Stone, C.S. 


Shreveport 


Bains, R. W. 
Nuttall, E. D. 


Tulsa 


Cheney, J. F. 
Day, C. E 
Freeman, L. R. 
Piety, in, G. 
Silverman, D. 
Sykora, G. Ie. 
Wolkov, D. 


Wichita 


Chapman, D. C. 
Haskell, H. B. 
Hickey, L. F. 
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Alamogordo-Holloman 


Benton, C. U. 
Boughton, E. M. 
Jaenke, M. G. 
Liston, D. H. 
Lynch, A. J., Jr. 
Weber, M. E. 


Albuquerque-Los Alamos 


Basore, B. L. 
Bidwell, C. H. 
Connell, J. C. 
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Jean, F. H. 
Kanneman, T. A. 
Koschmann, AE: 
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Melloh, A. W. 
Scharff, J. H. 
Stevens, R. R., Jr. 
Swain, G. R. 
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China Lake 


Ashbrook, F. M. 
Creusere, M. C. 
Herman, R. W. 
Poulson, W. A 
Zilmer, D. E. 


Fort Huachuca 


Eyjen, H. M. 
Lydon, D. A. 
Ternow, H. G. 


Los Angeles 


Adomian, G. 
Adrian, D. J. 
Albrecht, A. 
Albright, A. R. G. 
Alexander, M. A. 
Alpine, P. E. 
Ambrose, J. R. 
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Andrews, L. A 
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Ashby, R. M. 
Asheraft, W. D. 
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Brown, W. E., Jr. 
Bucher, Rees 
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